[Beowulf] zram usage

[Ellis H. Wilson III]

On 05/01/2014 03:57 PM, Prentice Bisbal wrote:
> On 05/01/2014 03:43 PM, Joe Landman wrote:
>> On 05/01/2014 03:33 PM, atchley tds.net wrote:
>>> Yes, but I don't want to buy a Dell just to get a NVMe unit. ;-)
>>
>> Not that I am trying to get the dude a Dell ... but think of it as
>> very expensive shrink wrap packaging ... :^
> On the bright side, a Dell box will be easier to open than a plastic
> clam-shell package.

Dell is the only one shipping the Samsung XS1715 to the best of my 
knowledge.  Please correct me if there is some other place which is. 
Samsung's the only one on the market with anything providing NVMe right 
now.  Should change within the year though.

But, as a word of caution, let's all beat this into our heads now: Flash 
is not RAM.  Flash is not RAM.  Flash is not RAM.  Not even slow RAM. 
(Ok, NAND Flash is not slow RAM -- maybe you could get away with saying 
NOR flash is slow RAM).  If it ain't byte-addressable, it ain't RAM.

That's not to say some very specialized (largely sequential, or aligned) 
things you do with RAM for the speed could also be done (almost as well) 
with Flash (particularly NVMe, and which would provide you higher 
capacities).  But do not, under any circumstances, expect to mmap flash 
or use flash as swap or do something equally terrible and expect it to 
perform reasonably.  Normal ram accesses do lots of fun un-aligned 
things because they can, and flash, NVMe or not, will kick and scream at 
that.

Getting back to zram, that's really interesting, and I had only heard of 
it in passing and not read up on it until now.  It looks like they added 
LZO/LZ4 support this past January, so you may have a wide enough variety 
of compressors now to find one that works for your use-case.

Then again, it looks like zram is just a staging mechanism (like a 
compressed ramdisk or the like) that creates a block device you can use 
to push or pull data from.  Could be way off on that.  So you'd have to 
think hard about how you want to use it.  Wouldn't be as simple as 
"allocate a bunch of memory and the kernel automagically compresses 
some/all in the background."

Best,

ellis

-- 
Ph.D. Candidate
Department of Computer Science and Engineering
The Pennsylvania State University
www.ellisv3.com