[Beowulf] Erlang as a language for Beowulf applications

[Andrew Piskorski]

On Mon, Sep 03, 2007 at 04:46:57PM -0400, H.Vidal, Jr. wrote:

> I am curious if anyone on this list has done any work
> with the language Erlang and/or considers it viable
> for scientific apps.

I don't know, but please do report back on what you find out!
I suspect you'll have to do some hands-on experimentation and
benchmarking yourself, and/or talk to the the Erlang/OTP experts
directly.

Erlang has its own native compiler now, in addition to its standard
interpreter, so perhaps its performance is now good enough for HPC
work...

There is quite a fair amount of info on Erlang out there, but, as you
probably noticed, it tends to be from a fairly Erlang-centric point of
view, which makes it harder to pre-evaluate before just actually
jumping in and using it.  For example:

1. HPC via message passing:

Message passing is of course very, very important in Erlang, but I'm
not sure how its performance and other characteristics compare to,
say, PVM and the various MPI implementations.  Presumably it's slower
than MPI, but how much?  With sufficient motivation, could you write
an Erlang driver for a high(er)-performance cluster interconnect, and
how much would that buy you?

2. Non-HPC messaging:

As of 4 or 5 years ago, I was told (by someone who tried both, I
believe) that neither of the two major open source message bus
toolkits, Spread and Ensemble, were really robust enough for
real-world production use.  Take that vague hearsay with a great big
grain of salt, of course - and even if it was true then, perhaps
they've since improved, I don't know.

But for it to do what it does, Erlang must have solved similar
messaging problems, probably in a different way.  (I'd guess using
Erlang's "expect failure and recover from it gracefully" approach in
some fashion.)  But I've never seen anyone do a compare/contrast of
Erlang vs. Spread, Ensemble, or other messaging approaches.

  http://www.spread.org/
  http://dsl.cs.technion.ac.il/projects/Ensemble/
  http://www.rabbitmq.com/

3. RDBMS:

Erlang/OTP ships with an interesting sounding "distributed database",
Mnesia, but I've never seen much info comparing/contrasting it to
other database systems out there.  What transaction model does it use
(MVCC, row locking, what?), what query features does it or doesn't it
support, etc.?

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/