[Beowulf] Good IB network performance when using 7 cores, poor performance on all 8?

[Brian Dobbins]

Hi John,

I am with Joe regarding looking at the interrupts.
>

  That was one of my initial thoughts, too, but I haven't really dug into
their states yet.  I'm going to start keeping an eye on them now.


> However, could this be a difference with the power management with the
> Redhat kernel?
> ie. when running on 8 cores you are tripping over some thermal threshold
> and causing a throttle back to a lower C-state?
>

  This is something I hadn't thought of but which makes a lot of sense, and
in my reply to Joe, you'll see we've got a power-saving state selected for
the PCIe ASPM.  I'm going to see if our guys can turn that off and whether
that helps, and I'll poke around some of the other tunable power options
soon, too.  I don't have root access and sit more on the application side
so I'm a bit rusty on all the buttons and dials, but I'll investigate a bit
and report back what I find.


> Can you give the kernel versions for both setups?
>

  The C5.5 kernel is 2.6.18-194
  The R6.2 kernel is 2.6.32-220

  There's a few more kernels in the mix now, but I'd like to check out a
few other things before I add to the confusion.  ;-)

  Thanks again,
  - Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140424/9a1488ab/attachment.html>