[Beowulf] Scaling issues on Xeon E5-2680

[cbergstrom at pathscale.com]

Do you have CPU affinity enabled? Is this omp + MPI? Which compilers and flags. Give more details about the software stack

  Original Message  
From: Jon Tegner
Sent: Sunday, February 28, 2016 22:28
To: beowulf at beowulf.org
Subject: [Beowulf] Scaling issues on Xeon E5-2680

Hi,

have issues with performance on E5-2680. Each of the nodes have 2 of 
these 12 core CPUs on SuperMicro SuperServer 1028R-WMR (i.e., 24 cores 
on each node).

For one of our applications (CFD/OpenFOAM) we have noticed that the 
calculation runs faster using 12 cores on 4 nodes compared to when using 
24 cores on 4 nodes.

In our environment we also have older AMD hardware (nodes with 4 CPUs 
with 12 cores each), and here we don't see these strange scaling issues.

System is CentOS-7, and communication is over FDR Infiniband. BIOS is 
recently updated, and hyperthreading is disabled.

Feel a bit lost here, and any hints on how to proceed with this are 
greatly appreciated!

Thanks,

/jon
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf