Lawrence Livermore National Laboratory and its partners AMD, Supermicro and Cornelis Networks have installed a new high performance computing (HPC) cluster with memory and data storage capabilities optimized for data-intensive COVID-19 research and pandemic response.
Funded by the Coronavirus Aid, Relief and Economic Security (CARES) Act, the “big memory” cluster, called Mammoth, will be used at LLNL to perform genomics analysis, nontraditional HPC simulations and graph analytics required by scientists working on COVID-19, including the development of antiviral drugs and designer antibodies.
“The ability of large-memory systems to integrate genomic analysis with large-scale machine learning for predictive modeling of therapeutic response will be important for accelerating the development of effective new therapeutics,” said Jim Brase, LLNL’s deputy associate director for Computing. “Mammoth will be integral for developing new tools to combat COVID-19, but also for fast response in a future pandemic.”
Mammoth comprises 64 nodes outfitted with second-generation AMD EPYC CPUs. Each node has two 64-core CPUs with 128 threads, features high-memory bandwidth and provides 2 terabytes (TB) of DRAM memory and nearly 4 TB of nonvolatile memory. The extra memory afforded by Mammoth is critical for COVID-19 researchers, who must sift through massive databases of information.
”It is exciting to see a direct connection between technology and the science being done to improve or even save lives,” said Dan McNamara, senior vice president, AMD Server. “AMD is proud to support the vital research being done by the team at LLNL with Mammoth in conjunction with our technology partners at Supermicro and Cornelis Networks.”
LLNL researchers have begun applying Mammoth to the genome of the SARS-CoV-2 virus that causes COVID-19, using the cluster to analyze how the virus evolves and simulating how its structure changes when they introduce mutations. Mammoth is helping reduce the time it takes to perform some types of genomic analysis from a few days to a few hours, Lab scientists said.
“This new system makes a big difference in how we prepare our jobs for calculations and in their performance,” said researcher Adam Zemla, who is using Mammoth for virus mutant detection, clustering, structural modeling and molecular dynamics simulations. “With Mammoth I can easily process COVID-19 genomes without the need to split datasets into smaller chunks, like I have had to do on previous machines.”
Mammoth’s large available memory resources and numerous cores are also aiding in the design of modified antibodies for improved binding against the virus, LLNL researchers said.
“In our workflow, we compute binding free energies with Rosetta Flex, a code that was memory-limited on other machines to 12 or 16 simultaneous calculations per node,” said LLNL computer scientist Thomas Desautels. “Mammoth enables us to run 128 Rosetta Flex calculations simultaneously on a single node, increasing our throughput by a factor of about eight. Using Mammoth, we can afford to execute many more Rosetta calculations and accelerate our search for SARS-CoV-2 antibody designs.”
Mammoth is the first production AMD-based HPC cluster connected with components from Cornelis Networks, a provider of purpose-built interconnects focused on HPC, high performance data analytics (HPDA), and artificial intelligence (AI). A spinout of Intel Corporation, Cornelis Networks launched in September with a scale-out interconnect portfolio that addresses the needs of demanding workloads and future innovative interconnect products.
“Cornelis Networks is thrilled to partner with AMD and Supermicro to deliver Mammoth to the researchers at Lawrence Livermore,” noted Phil Murphy, CEO of Cornelis. “Accelerating the path to a therapeutic for COVID-19 is emblematic of our company’s purpose to deliver innovative solutions that advance the mission of our customers to solve the world’s toughest problems.”
San Jose-based information technology company Supermicro provided the hardware for Mammoth, including the racks, servers, motherboards and overall system integration.
“Supermicro is excited to support an additional cluster at LLNL devoted to understanding the genetic makeup of the COVID-19 virus,” said Charles Liang, president and CEO of Supermicro. “Our 1U A+ Ultra installed in the Mammoth cluster leverages the system’s 64-core AMD EPYC CPUs, high-performance interconnects and large memory footprint, all in just one unit of rack space. The A+ Ultra platform’s performance and storage flexibility are ideal for the computationally intensive workloads used in running advanced simulations and research projects for these scientific challenges.”
Mammoth was procured through MNJ Technologies, an information technology firm headquartered in Illinois that served as the sourcing and logistics partner for the Supermicro HPC solution.
“MNJ and its employees are excited to be able to help make an impact regarding the health and wellness of the entire country,” said MNJ Technologies COO Paul Kozak. “This certainly provides a great sense of pride in the knowledge that MNJ played a small part in halting the advancement of this virus which has affected so many.”
For more, visit: https://hpc.llnl.gov/hardware/platforms/mammoth
AMD, the AMD logo, EPYC, and combinations thereof are trademarks of Advanced Micro Devices, Inc.
