[Beowulf] [gorelsky at stanford.edu: CCL:dual-core Opteron 275performance]

[Stuart Midgley]

We use numa tools to lock each process in an MPI job to a particular  
cpu and sets its memory affinity to the directly attached memory.   
The result is very effective.  We are able to get very reliable  
performance.

It required that someone in the group write a new mpirun to do all  
the placement (which required lots of hooks into the queue to  
determine exactly which cpu's to run on etc) and memory policies  
etc.  We also use the "job" kernel module to place each task in a job  
which means that a task can't escape our control.

To seg to another discussion, we are using SUSE and it is quite  
nice.  It is very progressive in the kernel modules it includes and  
works very well (it also happens to be the only linux that SGI  
supports).  Of course, running YAST completely blows away our  
customisations, so we avoid using any of the configuration tools and  
just modify the files ourselves.

Stu.



On 14/07/2005, at 0:56, Joe Landman wrote:

> Hi Mikhail:
>
>   If you use numactl, you should have control over processor  
> affinity for a particular process.  I am not sure how this ties in  
> to MPI though, so there may need to be some work there.
>
> Joe
>


--
Dr Stuart Midgley
sdm900 at gmail.com


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2361 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050714/d7e200b1/attachment.bin>