On Monday, Cerebras Methods unveiled its 13.5 million core Andromeda AI supercomputer for deep studying, studies Reuters. In accordance Cerebras, Andromeda delivers over one 1 exaflop (1 quintillion operations per second) of AI computational energy at 16-bit half precision.
The Andromeda is itself a cluster of 16 Cerebras C-2 computer systems linked collectively. Every CS-2 comprises one Wafer Scale Engine chip (typically known as “WSE-2”), which is at present the largest silicon chip ever made, at about 8.5-inches sq. and full of 2.6 trillion transistors organized into 850,000 cores.
Cerebras constructed Andromeda at a knowledge heart in Santa Clara, California, for $35 million. It is tuned for functions like giant language fashions and has already been in use for educational and business work. “Andromeda delivers near-perfect scaling through easy information parallelism throughout GPT-class giant language fashions, together with GPT-3, GPT-J and GPT-NeoX,” writes Cerebras in a press launch.
Cerebras
The phrase “Close to-perfect scaling” implies that as Cerebras provides extra CS-2 laptop models to Andromeda, coaching time on neural networks is decreased in “close to excellent proportion,” in response to Cerebras. Sometimes, to scale up a deep-learning mannequin by including extra compute energy utilizing GPU-based techniques, one would possibly see diminishing returns as {hardware} prices rise. Additional, Cerebras claims that its supercomputer can carry out duties that GPU-based techniques can not:
GPU not possible work was demonstrated by one in every of Andromeda’s first customers, who achieved close to excellent scaling on GPT-J at 2.5 billion and 25 billion parameters with lengthy sequence lengths—MSL of 10,240. The customers tried to do the identical work on Polaris, a 2,000 Nvidia A100 cluster, and the GPUs had been unable to do the work due to GPU reminiscence and reminiscence bandwidth limitations.”
Whether or not these claims maintain as much as exterior scrutiny is but to be seen, however in an period the place corporations typically prepare deep-learning fashions on more and more giant clusters of Nvidia GPUs, Cerebras seems to offer another strategy.
How does Andromeda stack up towards different supercomputers? Presently, the world’s quickest, Frontier, resides at Oak Ridge Nationwide Labs and might carry out at 1.103 exaflops at 64-bit double precision. That laptop price $600 million to construct.
Entry to Andromeda is accessible now to be used by a number of customers remotely. It is already being utilized by business writing assistant JasperAI and Argonne Nationwide Laboratory, and the College of Cambridge for analysis.
