GPUs are the backbone of AI computing, but as demand outstrips supply, cloud providers are getting creative.
Instead of waiting for another GPU, like Network world said they build custom chips to meet specific workloads, providing faster and more efficient computing while keeping costs under control.
The competition is heating up. At the Microsoft Ignite conference last week, the company introduced two new chips designed to increase performance for the Azure platform. All eyes are now on AWS as it prepares for its own silicon portfolio.
Why custom chips matter
GPUs have revolutionized tasks like training artificial intelligence models, but they aren’t always the best tool for the job. They come with significant drawbacks: high power consumption, intensive cooling needs, and currently a worldwide shortage. There is talk of Nvidia’s latest GPU inventory for the next 12 months.
Custom accelerators are stepping in to fill the gap. Mario Morales, vice president of analysts at IDC, emphasizes the growing importance of alternatives to GPUs: “These accelerators are becoming increasingly important in cloud infrastructure due to their excellent price/performance and cost/efficiency ratios, which lead to a better return on investment.”
AWS and Google have been rolling out their own chips for years—AWS with Trainium and Inferentia, and Google with Tensor Processing Units (TPU). However, Microsoft was late to the custom silicon trend. It wasn’t until last year that the company introduced its first proprietary Maia and Cobalt chips aimed at improving energy efficiency and handling AI workloads.
This year, Microsoft upped its game and introduced two new chips:
- Azure Boost DPU: Designed to optimize data processing by running its own operating system.
- Azure Integrated HSM: It focuses on security and stores encryption and signing keys securely in hardware.
Microsoft’s Azure Boost DPU is a step forward, but still lags behind competitors in the DPU space. Forrester senior analyst Alvin Nguyen notes that Google’s E2000 IPU, co-developed with Intel, and AWS’s Nitro system are already well established. Other cloud providers, including Nvidia with Bluefield chips and AMD with Pensando, are jockeying for position.
That said, Microsoft is making remarkable progress in infrastructure. The company announced new liquid cooling solutions for AI servers and an energy-efficient rack design developed in collaboration with Meta that can pack 35% more AI accelerators into each rack.
Security gets its own support
Security is another area where native silicon is making strides. Microsoft’s new HSM chip is a dedicated solution for encryption tasks that would traditionally require a combination of hardware and software. Nguyen notes that this approach reduces latency and improves scalability, making it an addition worth considering.
AWS and Google also use their own chips for security. AWS Nitro prevents core system processors from modifying firmware, and Google’s Titan creates a “secure root of trust” to verify system health.
Each provider has its own approach, Nguyen explains. “While Nitro provides a critical security feature ensuring that main system CPUs cannot update firmware in bare metal mode, Titan provides a hardware root of trust that creates a strong machine identity with which we can make important security decisions and verify system health.”
The future of custom chips in the cloud
The pressure on silicon itself is not slowing down. According to Alexander Harrowell, principal analyst at Omdia, it’s a logical move for hyperscalers to invest in these chips to reduce costs and increase efficiency.
With the growing demand for faster and more specialized computers, custom chips are a strategy for cloud providers to stay competitive. With overdrive innovation, the race to redefine cloud performance is just beginning.
(Photo by Unsplash)
See also: IBM wants Nvidia GPUs and AWS may be the answer
Want to learn more about cybersecurity and the cloud from industry leaders? Check out the Cyber Security & Cloud Expo in Amsterdam, California and London. Explore other upcoming enterprise technology events and webinars powered by TechForge here.