[Beowulf] memory bandwidth scaling

[John Hearns]

This machine is also prone to locking up (to the point it doesn't answer terminal keystrokes from a remote X11 terminal) when writing huge files back to disk.  I have not tracked this one down yet, it seems to be related to unmapping a memory mapped 10.5 Gb file.  A bit difficult to debug because when it is happening it isn't possible to look at what the machine is doing.


This is going to get more and more common as 'big memory' machines get more common.

In my last job I managed Altix Itanium machines with a terabyte of RAM and then SGI Ultraviolet.
Forgive me if I'm a bit fast and loose with terminology here. The Linux kernel just 'loves' to cache data. It will use a huge proportion of the free memory as cache.
This leading of course to the common question "My machine has run out of memory - look at what free is reporting to me'
Your friend here is  'watch  cat /proc/meminfo'  and show the user what the various types of memory allocation are doing.

Anyway, wiith a big memory machine you can have entire gigabyte sized files waiting to be flushed to disk - what happens if there is a power cut or a crash?
(I know I am being fast and loose here).
So look at the vm.dirty_background_ratio  and the vm.dirty_expire_centisecs

https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/

And also a plea for my hobby horse - not relevant here but up min_free_kbytes

#####################################################################################
Scanned by MailMarshal - M86 Security's comprehensive email content security solution.
#####################################################################################
Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Employees of XMA Ltd are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to company policy and outside the scope of the employment of the individual concerned. The company will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. XMA Limited is registered in England and Wales (registered no. 2051703). Registered Office: Wilford Industrial Estate, Ruddington Lane, Wilford, Nottingham, NG11 7EP