10-Tflops computer built off the shelf

[Eugen Leitl]

http://www.eet.com/at/news/OEG20021111S0037

10-Tflops computer built off the shelf

By Chappell Brown
EE Times
November 11, 2002 (5:20 p.m. EST)
 
LIVERMORE, Calif. . Lawrence Livermore National Lab is putting together a 
supercomputer that will boast nearly the same performance as the ASCII 
White system from IBM Corp. that the lab now uses but it promises to be 10 
times cheaper. Called Evolocity, the system will be the fastest clustered 
supercomputer in the world, according to Lawrence Livermore.

At around 11 teraflops, the system will not be as fast as the IBM machine, 
which ranks as one of the world's fastest computers. But it will be far 
easier on the pocketbook, thanks to the use of off-the-shelf components 
and the open-source Linux operating system.

The system has a few unique features that the lab says will facilitate 
applications performance, including a fast, custom-made network that taps 
into an enterprisewide file system. "We have been using the File Transfer 
Protocol over Gigabit Ethernet, but now we will be able to read files 
directly from any available disk," said Mark Seager, assistant department 
head for terascale systems at the lab.

"This network approach is nice because we can use a standard PCI slot on 
each processor node, which gives a 4.5-microsecond latency," he said, as 
opposed to 90-µs latency for Gigabit Ethernet.

The network uses bus host adapters on each node, supporting a 320-Mbyte 
transfer speed in one direction and 400-Mbyte bidirectional throughput. 
Each processing node is a server board from SuperMicro Inc. (San Jose, 
Calif.), built around the Intel E7500 chip set with two Xeon processors 
running at 2.4 GHz. The boards are linked by a network assembled by Linux 
Networx into a clustered system that will have 960 server nodes.

The file system, called Lustre, uses a client/server model. Large, fast 
RAM-based memory systems support a metadata center, and data is 
represented across the enterprise in the form of object-storage targets. 
"Being able to share data across the enterprise is an exciting new 
capability. It will allow more collaboration among research projects," 
Seager said. For example, workstations on the network running 
visualization programs can directly access data generated by Evolocity.

"The 1- to 10-teraflops processing range is opening up a revolutionary 
capability for scientific applications. It's qualitatively different from 
what we have been able to do before," said Seager. The difference lies in 
the number of variables the computer can process at the same time and in 
the resolution of the simulation. Not only are simulations much closer to 
realistic physical experiments, he said, but it takes far less time to 
converge to a reasonable approximation.

"This type of capability elevates computer simulation to the same level as 
physical experiment and theory, so it is going to allow us to do 
groundbreaking scientific work," he said. Also possible will be realistic 
simulation of three-dimensional systems evolving over time, or many 
variable solutions of quantum mechanics wave equations. The lab plans to 
use Evolocity first in simulating the time evolution of molecules, 
studying the aging process in materials and doing first-principle 
computations from the Schroedinger wave equation.

Clustered supercomputers using off-the-shelf components have been pursued 
by smaller research groups. Evolocity shows that major labs can also reap 
benefits from the approach.