Tech titan Amazon is working to step out of Nvidia’s shadow with custom “Trainium” chips designed specially for machine learning as billions of dollars are poured into artificial intelligence (AI).
Amazon subsidiary Annapurna Labs, founded in Israel and acquired by Amazon in 2015, was testing the longevity of its latest generation Trainium processors at its Austin, Texas facility during a recent visit by AFP.
Texas is emerging as a US tech world El Dorado, luring investments with cheap energy, relaxed regulations, tax incentives and reasonably affordable real estate for massive data centres.
Amidst a deafening roar, UltraServers packed with 144 of the Trainium AI-accelerator chips were being put through their paces at Annapurna in a routine check prior to delivery.
Related
After years of relying on suppliers for chips, the e-commerce powerhouse’s Amazon Web Services (AWS) cloud computing unit began designing its own, acquiring Israeli startup Annapurna Labs in 2015.
First came Graviton and Inferentia chips in 2018, the former for general cloud computing and the latter for powering AI models.
The first Trainium debuted in 2020, followed by a second generation that touted a big boost in performance.
Trainium 3 chips put into action in December are touted as doubling the capabilities of the second generation despite being smaller than a credit card.
Kristopher King, head of the Annapurna lab in Austin, contended that the latest Trainium chips can cut the cost of developing and running generative AI models by as much as 40 percent compared to using graphics processing units (GPUs) that are now deemed the “gold standard” for AI.
Failure not an option
Along with pricing Trainium chips competitively, AWS is out to make reliability a selling point since data centres need to operate non-stop for long stretches at a time.
AI development requires hundreds of thousands of chips operating simultaneously for weeks, according to Annapurna head of engineering Mark Carroll.