Fusion HCI Performance Boost for AI Apps


Fine-tuning and inferencing with foundation models used in AI applications increasingly requires faster and more powerful hardware as AI foundation models grow ever larger.  In response to this need, IBM Storage Fusion HCI System released a new GPU server option earlier this month called the G02 that gives a performance boost for AI applications.

The new G02 GPU server is based on the Lenovo SR650 V3.  The configuration used in the G02 includes a pair of Intel Xeon Gold 6418H 24-core "Sapphire Rapids" processors.  This processor model includes Intel Deep Learning Boost, designed to accelerate AI use cases by extending the Intel AVX-512 instruction extension with a new instruction that increases deep learning inference performance.

But the big boost provided by the G02 for AI apps comes from the three NVIDIA A100 80GB GPU PCIe adapter cards in each G02 server.  This newer version of the A100 has up to 1.25x higher AI inference performance over the A100 40GB that was used in the original Fusion HCI GPU server, the G01.  And as the name suggests, these newer A100 GPU adapters also have twice as much RAM.  This larger amount of RAM is critical for loading larger foundation models that need to be loaded in their entirety to get the best performance.

In addition to these significant GPU and CPU resources, there is 512GB of RAM configured as 16x 32GB DIMMs.  That works out to be one DIMM in every memory channel for maximum memory performance.  The CoreOS operating system used by Red Hat OpenShift is installed on two 960GB M.2 NVMe OS drives in a RAID 1 configuration for redundancy.  Local storage is provided by a pair of 2.5 inch Samsung PM1655 3.2TB SAS 24Gb SSDs that also use RAID 1 mirroring for redundancy.

Like every other server in a Fusion HCI rack, the G02 server has redundant network connections to the OpenShift application network. These are provided by an NVIDIA ConnectX-6 LX dual-port 25GbE PCIe NIC. And the G02 also has redundant connections to the high-speed storage network using an NVIDIA ConnectX-6 DX dual-port 100GbE PCIe NIC.

Designed to boost the performance of AI applications with powerful GPU and CPU technology, the new Fusion HCI G02 GPU server is the ideal option to add to an IBM Storage Fusion HCI System!

To learn more about how Fusion HCI uses its GPU resources like the G02 in support of AI workloads, check out this page: https://ibm.github.io/storage-fusion/watsonx/overview

Previous post: A New Direction for IBM Storage



The opinions expressed in this post are those of the author.

Comments

Post a Comment

Popular posts from this blog

Inside the Storage/Compute Servers of IBM Spectrum Fusion HCI

Image
Previously, I provided an overview of the IBM Spectrum Fusion HCI hardware followed by posts giving an overview of the different server and switch components that go into that system. Most of the servers in an IBM Spectrum Fusion HCI system will be storage/compute servers, so you can think of these servers as the workhorses of the system. In this post, I take a closer look at the CPU, memory, storage, and network elements inside these storage/compute servers. CPU and Memory A storage/compute server is built from a Lenovo SR645 server that is one rack unit (1U) high.  (A rack unit is an industry standard that translates to 1.75 inches; an industry standard data center rack used to hold servers and switches is 42U tall.)  Inside the server are two processor sockets that are each populated with an AMD EPYC 7302 processor.  The EPYC 7302 is a 16-core, 32-thread, processor that operates with a base clock speed of 3.0 GHz.  With two of these process...
Read more

Fusion HCI Adds 8x the GPU Power

Image
IBM Storage Fusion HCI System has released a new GPU server today that takes a huge step forward in the support of AI applications.  The new server, called the G03, can be configured to have as many as 8 double-wide GPU adapter cards .   Fine-tuning and inferencing with foundation models used in AI applications increasingly requires faster and more powerful hardware, and the Fusion HCI G03 meets that need head on! The tremendous boost provided by the G03 for AI apps comes from having up to 8 of the  NVIDIA L40S  GPU PCIe double-wide adapter cards inside a single server.  The L40S is powered by the NVIDIA Ada Lovelace Architecture and is designed for generative AI and LLM training and inferencing.  It contains fourth generation tensor cores and the latest transformer engine that puts those tensor cores to good use. The design of the G03 provides flexibility for deploying GPU resources.  Many organizations will order the G03 with a full complement o...
Read more