[Beowulf] Newbie Question: Racks versus boxes and good rack solutions for commodity hardware

[Vincent Diepeveen]

On Dec 15, 2008, at 6:07 AM, arjuna wrote:

> Hi Vincent:
>
> I am a believer in beginners mind.  Some of the things that i am  
> doing like indian classical music, classical drawing, python  
> programming are all things i got inspired to do and dived right  
> into. And the inspiration is usually right on and after several  
> years of doing it, one achieve a masterful level. This does entail  
> a lot of passion, inspiration and the courage to go ahead and do it.
>

"My education and training at art academy was one big course in  
psychology"
     Maud Veraar (marketing manager highq.nl)

After having negotiated with a couple of hundreds of graphics  
designers past 10 years,
i definitely recognize in the start paragraph you write here that you  
also have strong feelings
in that direction that you admire people who have that social gift  
that these creative professionals
all have in such abundance. Basically that means that a social part  
of the brain has developed itself
very well where most on this list need more than a paid mission to  
Mars to discover a part of it.

Wanna bet they gonna find courageous volunteers for that single  
ticket to Mars?

> I have been bitten by the animation bug, and this has something to  
> do with the fact that i am currently deeply immersed in a several  
> year training in the classical drawing for classical realist  
> painters. Animators study drawing, but probably not in this
> depth. There is a huge synergy as you would know between drawing  
> and animation which requires good visual mental faculties that are  
> developed by drawing.


Already visited Amsterdam?

The best painters from what we call 'the golden age', Netherlands was  
world power number 1 for a year or 100 a year or 300 ago and the huge  
money that this produced the dutch used especially to let the best  
painters on the planet make paintings of a quality that todays  
artists do not have (as in todays society a lot of additional factors  
determine the success of artists rather than just their work -  
nowadays you also need to be verbal strong and sell your product to  
the audience, maybe that reduced the level of todays art quite a bit.  
The best museums on the planet to see this brilliant type of realist  
painting is in the National Museum (and next to it at 100 meters away  
is the Van Gogh Museum) located in Amsterdam.

This is worlds best and a really LARGE collection of paintings. In  
fact the huge museum just shows a very tiny part of all the paintings  
they posses.

Because of the beauty of those paintings donors buy slowly at  
auctions paintings to get them back where they were made.
Some paintings get sold for tens of millions of euro's. A price  
really high if you consider the tens of thousands of paintings
made in those centuries that survived until today.

>
> So having become inspired by the animation, i have been researching  
> it avidly. As an artist I am interested not in making short fun  
> clips but eventually entire movies. I realize this is a large  
> undertaking, but thats what i do...So is vocal indian classical  
> music, kung fu and classical drawing.
>
> In researching Pixar which makes these kind of movies, i found they  
> use farms of machines for rendering. Therefore the idea

Yeah all those art academy guys are worlds best marketing managers,  
specialized in selfpromotion.
I lack that gift, as Greg Lindahl will happily confirm for you here,  
though he never commented on this list
what he finds from president-elect Obama's energy speech that is at  
his website already for a while.

A small problem for artists is of course that if 1 out of a 10k  
artists is one of the better art artists on planet earth,
which regrettably means that each year 9999 ex-students need a job as  
they lack the talents of a Rembrandt van Rijn,
or a Vincent van Gogh.

I'm quite sure Pixar bought that hardware to provide that huge base  
of designers some computing power to produce
what they need. Where would Pixar get the money to pay for so many  
people who basically would be jobless based upon skillset?

Making animations is not so easy as most people guess.
Very few are good at it. Most complicated is designing the characters.

I remember that a very experienced designer who had made a female  
character, which took him quite long to make,
as it had like tens of thousands of rectangles, that i asked him the  
question: "What is the biggest difference between
a human being and that ape that you drew?".

After i commented that the forehead of a human being is further  
forward than from an ape, he agreed with me and corrected
that. Then it looked like a human. The GUI programmer then toyed for  
a week with that character to get the boobs bigger ;)

After that it looked like something.

I quote this because this designer already is one of the better  
designers. Designing human beings, especially the head,
it takes a designer up to a month to create.

All he (amazingly very few 'she's are designing 3d graphics) needs  
during all that time is a real cheapo PC.

There is plenty of system time available to create the animations.  
Designing the graphics is what eats all time and most money.

Note i hear the word '3d studio max' just too much. Yeah sure, me as  
big layman designer also toyed a year or 10 ago in 3d studio max and  
i still would do it. For big movies and most big game companies other  
(a bit older) products
get used (lightwave, maya etc). 3d studio max is basically popular at  
art academies (maybe because of licensing).

That will change slowly of course.

You cannot convert animations easily from 1 product to another.
Really a big problem that is, as a designer trained in product A will  
just not manage (in 99% of the cases)
to work with another software product.

> was planted in my brain, and being a good computer programmer I  
> decided to ride the learning curve(and i know how to ride these  
> wild horses given the amount of learning required in the classical  
> arts and the time and patience it takes) and make me a beowulf  
> which is the cheapest fastest machine.
>

In this mailing list the word 'good computer programmer' means  
basically that you don't need a cluster at all.
All you need in that case is a Tesla card which has a price of say  
1300 dollar or so at newegg and render with your
own 3d engine the animations.

Yeah that engine is better than what lightwave has (not surprising).  
Especially for light effects these are tricky to get right.

That's what we did do (not in cuda, simply at the pc).
A single core P4 can render a second or 10 of animations of an entire  
scene within a few minutes.
That's a 3d engine in C/C++ code, not even using SSE2 assembler.

Let alone using 240 cores (as everything is single precision float  
anyway) of a graphics card.

That last would really kick butt. A programmer can port such code to  
such hardware platform.

A single GPU is so so much faster for this.

Also it has enough ram. When a few parts of animation rendered, you  
can stream that from card to the
PC where the pc processor can compress it to some sort of lossy high  
quality mp4 format and put it on the disk.

Not a single cluster with quadcores can compete against this.

Especially not against its price.

> Made from commodity components it would be cheaper than high end  
> machines (hence the relevance to this list in case you are  
> wondering) it would eventually give me the power (having ridden the  
> curve over time, not tomorow) to do the type of rendering that the  
> big boys are doing.
> If Pixar is using render farms then i assume that is a good well  
> thought out way to go for eventually creating full length animated  
> movies. Also i am interested in special effects.
>
> The whole logic of the beowulf is to use cheap commodity machines  
> to make super computers(to put is a bit simplistically) So
> why would one use an expensive stand alone machine with eventually  
> limited capacity? If it were possible to make full length, cutting  
> edge animated movies using stand alone machines then the boys who  
> actually make them would be using them for their productions right?
>

With a simple box you can already produce an entire movie handsdown  
within a few hours.

If you want to do more than being sysadmin of the system, namely be a  
programmer, you might want
to do the rendering at some sort of Tesla type hardware. That's a tad  
of programming of course, but
it goes completely realtime in such case.

If it is ok for pilots in training that manner, it must be ok for you  
as well.

A big cluster with expensive network for the i/o is going to be very  
expensive compared to the box that
can do the job.

>
> Hello Arjuna,
>
> I'm a bit interested you mention this. When i negotiate about  
> animations, which sometimes already were edits from major movies,
> the hardware used is nothing more than a simple PC to produce it.  
> Animation Designers work for small amounts of money,
> so paying for a lot of hardware, more than some fast PC or fast  
> macintosh, with 2 very good big TFT's (apple offers some fantastic
> dual set of TFT's - though quite expensive for a design studio maybe).
>
> It sure takes a couple of minutes to render animations in high  
> resolutions, yet i'm quite amazed you need more hardware for this,
> yes even a cluster.
>
> Isn't some 16 core Shanghai box with a lot of RAM already total  
> overpower for this?
>
> Can you explain where you need all that cpu power for?
>
> Best Regards,
> Vincent
>
> Also any thoughts on racks versus piles of PCS.
>
> A lot of the posts on the internet are old and out of date. I am  
> wondering what the upto date trends are in racking commodity  
> computers to create beowulf clusters. What should i be reading?
>
> -- 
> Best regards,
> arjuna
> http://www.brahmaforces.com
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>
>
> -- 
> Best regards,
> arjuna
> http://www.brahmaforces.com
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf