While central processing units and graphics processing units serve different roles in AI training and inferencing, both roles are vital to AI workloads.
CPUs and GPUs were both invented long before the AI era. But each has found new purpose as the robots conduct more of our day-to-day business.
Each has its tradeoffs. Most CPUs are less expensive than GPUs, and they typically require less electric power. But that doesn’t mean CPUs are always the best choice for AI workloads. Like lots of things in life, it depends.
Two Steps to AI
A typical AI application involves a two-step process. First training. Then inferencing.
Before an AI model can be deployed, it must be trained. That could include suggesting which movie to watch next on Netflix or detecting fake currency in a retail environment.
Once the AI model has been deployed, it can begin the inferencing process. In this stage, the AI application interfaces with users, devices and other models. Then it autonomously makes predictions and decisions based on new input.
For example, Netflix’s recommendation engine is powered by an AI model. The AI was first trained to consider your watching history and stated preferences, as well as to review newly available content. Then the AI employs inferencing—what we might call reasoning—to suggest a new movie or TV show you’re likely to enjoy.
AI Training
GPU architectures like those found in the AMD Instinct MI325X accelerator offers highly parallel processing. In other words, a GPU can perform many calculations simultaneously.
The AMD Instinct MI325X has more than 300 GPU compute units. They make the accelerator faster and more adept at both processing large datasets and handling the repetitious numerical operations common to the training process.
These capabilities also mean GPUs can accelerate the training process. That’s especially true for large models, such as those that underpin the networks used for deep learning.
CPUs, by contrast, excel at general-purpose tasks. Compared with a GPU, a CPU will be better at completing sequential tasks that require logic or decision-making. For this reason, a CPU’s role in AI training is mostly limited to data preprocessing and coordinating GPU tasks.
AI Inferencing
However, when it comes to AI inferencing, CPUs play a much more significant role. Often, inferencing can be a relatively lightweight workload, because it’s not highly parallel. A good example is the AI capability present in modern edge devices such as the latest iOS and Android smartphones.
As mentioned above, the average CPU also consumes less power than a GPU. That makes a CPU a better choice in situations where heat and battery life are important.
However, not all inferencing applications are lightweight, and such workloads may not be appropriate for CPUs. One example is autonomous vehicles. They will require massive parallel processing in real-time to ensure safety and optimum efficiency.
In these cases, GPUs will play a bigger role in the AI inferencing process, despite their higher cost and power requirements.
Powerful GPUs are already used for AI inferencing at the core. Examples include large-scale cloud services such as AWS, Google Cloud and Microsoft Azure.
Enterprise Grade
Enterprises often conduct AI training and inferencing on a scale so massive, it eclipses those found in edge environments. In these cases, IT engineers must rely on hugely powerful systems.
One example is the Supermicro AS -8125GS-TNMR2 server. This 8U behemoth—weighing in at 225 pounds—can operate up to eight AMD Instinct MI300X accelerators. And it’s equipped with dual AMD EPYC processors, the customer’s choice of either the 9004 or 9005 series.
To handle some of the world’s most demanding AI workloads, Supermicro’s server is packed with an astonishing amount of tech. In addition to its eight GPUs, the server also has room for a pair of AMD EPYC 9005-series processors, 6TB of ECC DDR5 memory, and 18 hot-swap 2.5-inch NVMe and SATA drives.
That makes the Supermicro system one of the most capable and powerful servers now available. And as AI evolves, tech leaders including AMD and Supermicro will undoubtedly produce more powerful CPUs, GPUs and servers to meet the growing demand.
What will the next generation of AI training and inferencing technology look like? To find out, you won’t have to wait long.
Do More:
- Check out AMD Instinct MI300 accelerators
- Get tech specs on Supermicro’s 8U rackmount GPU server