Amazon launched a new tool to make it easier to run artificial intelligence (AI) applications on its own chips and challenge Nvidia's position in the market. Named TorchNeuron, the feature allows developers to port applications created with PyTorch to Amazon Web Services' (AWS) Trainium processors with only a configuration change, without needing to rewrite the code.
The initiative targets one of Nvidia's main competitive advantages: the software ecosystem that helps keep its chips as the default choice for many companies that develop AI.
PyTorch is a free, open-source tool used to develop, train and run AI models. According to Amazon, approximately 85% of AI applications are developed with the framework.
Nvidia consolidated its presence in this market over the years through investments in CUDA, its software platform that enables AI applications to run on its graphics processors. As a result, a large portion of the programs developed with PyTorch is already adapted to work with the company's hardware. TorchNeuron aims to reduce this dependency and make migration to Amazon's chips simpler.
Amazon wants to make switching AI chips easier
- The tool was adapted from PyTorch and leverages the knowledge developers already have about the framework. In practice, the proposal is to enable existing applications to run on Trainium chips without requiring code to be rewritten;
- Amazon is not the first company to try to make alternative hardware easier to use. Developers can also adapt PyTorch applications for AMD graphics processors and Apple chips with relatively minor configuration changes;
- The company’s claimed advantage is integration with AWS’s entire cloud computing infrastructure. This way, startups could develop prototypes, train models, and put applications into production within the same ecosystem;
- Colin Brace, vice president of software at Annapurna Labs, Amazon’s unit responsible for developing its own chips, argues that this integration offers advantages over providers specializing in AI infrastructure;
- “You’re building on AWS. You’re not betting on a single-purpose compute provider and racing to build everything else around it,” the executive told TechCrunch;
- Amazon also maintains that using the same type of chip for training and inference can improve operational cost efficiency. Training is the stage in which the model learns from data, while inference involves using the already-trained model to perform tasks, such as answering questions or processing audio.
Read more:
- Amazon to invest R$ 5.2 billion in data center communities in the US
- Amazon cuts jobs in retail division in new round of layoffs
- Amazon negotiates acquisition of AI startup valued at US$7 billion
Startup reports increased productivity with Amazon chips
Amazon cites Deepgram, a company specializing in AI for speech processing, as an example of the potential benefits of the switch. According to Brace, the company began using Trainium chips and was able to process 70% more audio per dollar while maintaining the accuracy required for production operations.
Amazon uses this account to argue that the choice of AI infrastructure should not depend solely on the individual price of chips or the cost per unit of processing. The company argues that performance, training, inference, and cloud services must all be considered when calculating the total cost of operations.
Even so, AWS's offering will have to compete with an established ecosystem. Nvidia does not rely solely on the performance of its processors: its software, available tools, and developers' familiarity also help sustain its market position.
Amazon wants to make PyTorch less dependent on Nvidia
The company's plan goes beyond making it easier to migrate to its own chips.Amazon also intends to contribute changes to PyTorch’s main project to reduce the framework’s automatic association with certain hardware manufacturers.
The idea is to make the development environment more independent of the chips used, allowing different companies to offer alternatives without requiring complex software adaptations.
The strategy may broaden the options available to developers and companies that need substantial computing capacity to train and run AI models. At the same time, Nvidia may face greater competition not only from processor manufacturers, but also from cloud providers that develop their own chips.
Amazon, however, still needs to demonstrate that the ease promised by TorchNeuron translates into practical advantages for different applications. Abstracting away the differences between chips is precisely one of the functions of frameworks such as PyTorch, and changes at this layer may bring configuration and compatibility challenges.
If the initiative can significantly reduce developers’ dependence on Nvidia’s ecosystem, its effects may go beyond the direct competition between chip manufacturers.
“The real story is to eliminate the lock-in that prevented developers from having a genuine choice,” Brace said.
Rodrigo Mozelli is a journalist who graduated from São Paulo Methodist University (UMESP) and is currently a writer at Olhar Digital.
Watch movies and TV shows
Subscribe to Amazon Prime to watch popular movies and TV shows, including award-winning Amazon Originals. Amazon Prime also includes fast, FREE delivery on thousands of eligible items, over 2 million songs without ads, and much more. Click here and start your FREE 30-day trial!
