Dolphin Wulfkit

[Patrick Geoffray]

joachim wrote:

>>>Lazy memory unlocking breaks correct programs that use memory
>>>dynamically.  It means that the programmer must program in a
>>>restricted way with memory that is subject to send/receive.
>>
>>Why is that ? It's possible to implement lazy memory unlocking without 
>>imposing any constraint on the application.
>
> You need to modify the OS for this, as I see it, MPI alone can not do 
> it correctly. Or just turn of paging... How does GM ensure un-pinning
> when memory is free'd?  

It's what we do in MPICH-GM, without OS support. We provide our malloc 
code (that is the same than the malloc used in the Linux LibC, Doug 
Lea's malloc) and we catch the calls to munmap() and sbrk() with a 
negative argument. In these 2 stubs, we look up the pages in the 
registration cache and eventually deregister them, before calling the 
real munmap() and sbrk(). It's totally safe to free registered pages, if 
they stay in the malloc pool, what you need to do is deregister them 
before the malloc give them back to the kernel.
You can look at it in mpid/ch_gm/gmpi_malloc.c and 
mpid/ch_gm/mallocaux.c for implementations detais, it's a dozen of lines.

> - you didn't sleep while I was talking at CAC'02, did you? ;-)

No, I was eating my bagel, and the cheese was falling from my spoon, it 
was a mess :-)

Patrick

----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      patrick at myri.com
|   Myricom, Inc.                http://www.myri.com
|   Cell:  865-389-8852          685 Emory Valley Rd (B)
|   Phone: 865-425-0978          Oak Ridge, TN 37830
----------------------------------------------------------