If you develop AI and HPC solutions, you’ll want to know about the most recent release of AMD ROCm software, version 6.2.
ROCm, in case you’re unfamiliar with it, is AMD’s free and open software stack. It’s aimed at developers of artificial intelligence and high-performance computing (HPC) solutions on AMD Instinct accelerators. It's also great for developing AI and HPC solutions on AMD Instinct-powered servers from Supermicro.
First introduced in 2016, ROCm open software now includes programming models, tools, compilers, libraries, runtimes and APIs for GPU programming.
ROCm version 6.2, announced recently by AMD, delivers 5 key enhancements:
- Improved vLLM support
- Boosted memory efficiency & performance with Bitsandbytes
- New Offline Installer Creator
- New Omnitrace & Omniperf Profiler Tools (beta)
- Broader FP8 support
Let’s look at each separately and in more detail.
LLM support
To enhance the efficiency and scalability of its Instinct accelerators, AMD is expanding vLLM support. vLLM is an easy-to-use library for the large language models (LLMs) that power Generative AI.
ROCm 6.2 lets AMD Instinct developers integrate vLLM into their AI pipelines. The benefits include improved performance and efficiency.
Bitsandbytes
Developers can now integrate Bitsandbytes with ROCm for AI model training and inference, reducing their memory and hardware requirements on AMD Instinct accelerators.
Bitsandbytes is an open source Python library that enables LLMs while boosting memory efficiency and performance. AMD says this will let AI developers work with larger models on limited hardware, broadening access, saving costs and expanding opportunities for innovation.
Offline Installer Creator
The new ROCm Offline Installer Creator aims to simplify the installation process. This tool creates a single installer file that includes all necessary dependencies.
That makes deployment straightforward with a user-friendly GUI that allows easy selection of ROCm components and versions.
As the name implies, the Offline Installer Creator can be used on developer systems that lack internet access.
Omnitrace and Omniperf Profiler
The new Omnitrace and Omniperf Profiler Tools, both now in beta release, provide comprehensive performance analysis and a streamlined development workflow.
Omnitrace offers a holistic view of system performance across CPUs, GPUs, NICs and network fabrics. This helps developers ID and address bottlenecks.
Omniperf delivers detailed GPU kernel analysis for fine-tuning.
Together, these tools help to ensure efficient use of developer resources, leading to faster AI training, AI inference and HPC simulations.
FP8 Support
Broader FP8 support can improve the performance of AI inferencing.
FP8 is an 8-bit floating point format that provides a common, interchangeable format for both AI training and inference. It lets AI models operate and perform consistently across hardware platforms.
In ROCm, FP8 support improves the process of running AI models, particularly in inferencing. It does this by addressing key challenges such as the memory bottlenecks and high latency associated with higher-precision formats. In addition, FP8's reduced precision calculations can decrease the latency involved in data transfers and computations, losing little to no accuracy.
ROCm 6.2 expands FP8 support across its ecosystem, from frameworks to libraries and more, enhancing performance and efficiency.
Do More:
- Read the AMD ROCm 6.2 release notes
- Browse and download AMD ROCm documentation
- Check out Supermicro systems powered by AMD Instinct MI300 accelerators, which can be developed for using ROCm
Watch the related video podcast: