[Beowulf] Intel buys QLogic InfiniBand business

Mon Jan 23 16:40:13 PST 2012

On Jan 24, 2012, at 1:03 AM, Joe Landman wrote:

> On 01/23/2012 06:24 PM, Vincent Diepeveen wrote:
>>
>> On Jan 24, 2012, at 12:02 AM, Joshua mora acosta wrote:
>
> [...]
>
>> Nanosecond latency of QPI using 2 rings versus something that has a
>> latency up to factor 1000 slower
>> with the pci-e as the slowest delaying factor.
>>
>> Doing cache coherency over that forget it.
>
> Hear that Shai F?  Stop work on vSMP now, cause Vincent says it can't
> work!!!
>
> More seriously, with this acquisition, I could see serious contention
> for ScaleMP.  SoC type stuff, using IB between many nodes, in  
> smaller boxen.
>

That would be some BlueGene type machine you speak about that intel  
would produce with a low power SoC.

This where at this point the bluegene type machines simply can't  
compete with the tiny processors
that get produced by the dozens of millions.

"The tiny processors have won"
    Linus Thorvalds

Intel has themselves a second law of Moore. You can google for it.  
Every new generation of factory that
can produce this machine with double the number of transistors, that  
factory also is 2x more expensive.

A few years ago intel projected that by 2020 building a single  
factory would have a cost of 20 billion dollar.

Now Obama might contribute to this by overspending 40-50%, more  
overspending than the overspending of
Greece, Spain, UK and Portugal combined.

So that will cause massive inflation, which will hurt the poor most,  
and it sure will help the 2nd law of Moore become sooner a reality
rather than later; yet if we move away from politics to money and  
mass production;
i hope you realize that a few HPC cpu's won't pay back for 20 billion  
dollar.

In short only cpu's that get mass produced can.

A good example of massproduced processors are gpu's.

If we look at the leading gpu's, which have by now thousands of  
cores, there is no way to compete with that with SoC's.

What's price of producing 1 gpu versus 200 SOC's with a small core?

Furthermore intel never really could compete in the SOC world so far  
with the low power cpu's that get produced by the billion a year,
so betting on that would be quite surprising, though not impossible  
gamble.

Intel always has been good in low latency designs. yet obviously  
further integration of logics into the cpu means of course you also
need a capable ethernet chip in your cpu. Qlogics can provide that.

Mass produce half a billion of those and then it's cheaper to buy a  
company with such technology than to pay royalties.

Another HPC problem with the bluegene type designs:

all those soc's basically spread the calculation power over a bigger  
area than 1 big power eating chip will.
Bigger area means bigger distance to transfer massive data, and  
that's in itself a very expensive thing.

Overall seen bluegene machines never really had a low power usage,  
despite some stupid professors shouting that.
Per gflop it always was never the performance king; they just  
compared with total hopeless type designs and IBM usually
delivered in time, something that is very important in HPC as well.

IMHO the only reason bluegene could be competative is because it was  
fighting dinosaur type HPC cpu's.

Now SoC's might be mighty interesting in the gamersworld and in the  
telecom to build new phones with,
wich makes it mighty interesting for intel to produce those  
dirtcheap, and maybe even put a more capable ethernet
chip on it, again dirtcheap; as for the HPC world i don't see it  
happen that this SoC can compete anyhow with a gpu or even CPU.

Better write some code in CUDA or OpenCL i'd argue.

Latest AMD gpu the HD Radeon 7970, it is delivering 1 teraflop or so?

With soon a 2 gpu version coming on 1 card that's gonna deliver close  
to 2 Tflop a card, double precision yes.
Multiply by 4 for single precision. 8+ Teraflop single precision.

For a couple of hundreds of dollars. Nvidia will undoubtfully follow  
with their 1 teraflop gpu.

If take a washing machine and pack it with cheapo socks, creating a 2  
Tflop machine, do you guess you can SELL that for a couple of
hundreds of dollars?

Just transport costs already will be more expensive than a single gpu  
card...

Intel cannot compete with that in HPC for the stuff that needs  
bandwidth and doesn't care for latency. as at a new proces technology,
they first go produce a few FPGA cpu's, and after that they produce  
worlds fastest CPU. So there is simply no window in
time to use the latest proces technology for a HPC vector type chip.  
That's why AMD-ATI and Nvidia will win that contest handsdown.

And we sure hope intel will keep selling its cpu's very well, which  
if it is the case means that this won't change.

After all they already make cash on majority of supercomputers as  
each node also usually has 2 Xeon cpu's which go for a multiple of  
the price
of the GPU that's in the box...

>
>>> In any case, by acquiring their IP it is a step forward towards SoC
>>> (System on
>>> Chip). A preliminary step (building block) for the Exascale
>>> strategy and for
>>> low cost enterprise/cloud solutions.
>
> Yes.
>
>> Not with intel. Intel sells fast equipment yet it has a huge price
>> always,
>> about the opposite of infiniband which is a dirt cheap technology.
>
> Must use Shakespeare for this takedown:  Methinks thou dost protesteth
> too much ...
>
>>
>> I guess we must see this much simpler. At such a giant as intel,
>> paying a bit over 100 million is peanuts.
>> Probably less than what they would need to pay for royalties to a
>> manufacturer owning a bunch of patents
>> in the ethernet NIC area; the HPC intel gets 'for free'.
>
> So ... exactly what are the existing intel 10GbE NIC's then ... Swiss
> Cheese?  I see a fair number of vendors licensing Intel's IP, or, more
> to the point, using Intel silicon (hint: this might be a good  
> reason for
> the acquisition) to build their stuff...
>
>> Allows them to produce maybe a 10 gigabit ethernet NIC dirt cheap
>
> ... which they have been doing for years ...
>
>> without needing to pay royalties to qlogic.
>
> ... not sure they were, but its possible Qlogic has 10GbE IP that  
> Intel
> licenses, but this transaction was about ... Infiniband ...
>
> [...]
>
>> meanwhile ethernet is total crucial to have low latency for the
>> financial world, as they can make dozens of billions a year by being
>> faster
>> than others at exchanges.
>
> Errr ... given that this is one of our core markets, don't mind if I
> note that latency is critical to these players, so proximity to the
> exchange, and reliable and deterministic latency is absolutely  
> critical.
>   There are switches that are doing 300ns port to port in the Ethernet
> space now.  With the NICs, you are looking in the 2-ish microsecond
> regime.  These are not cheap.
>
> Compare this to QDR.  1 microsecond +/- some.
>
> Which has lower latency?
>
> There are many reasons why exchanges (mostly) aren't on IB.  A few of
> them are even valid technical reasons.  Historical momentum, and
> conservative approaches to new technology rank pretty high.  So  
> does the
> inability to generally export IB far and wide.  And the complexity of
> the stack.  Ethernet is (almost) plug and play.  Its just a network.
>
> IB is sort of kind of plug, install OFED, and play for a while over
> IPoIB until you can recode for some of the RDMA bits.  And don't  
> try to
> run file systems and other things with lots of traffic over IPoIB.  It
> leaks and gradually you will catch some cool ... surprises.
>
> Honestly, its a shame that IPoIB never really got the attention it
> deserved like the other elements of the IB stack did.  Getting a rock
> solid IP implementation atop a fast/low latency net could have driven
> many design wins outside of HPC.  And would have been a gateway
> drug^H^H^H^Htechnology for using the other stack elements.
>
>
>
> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics Inc.
> email: landman at scalableinformatics.com
> web  : http://scalableinformatics.com
>         http://scalableinformatics.com/sicluster
> phone: +1 734 786 8423 x121
> fax  : +1 866 888 3112
> cell : +1 734 612 4615
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf