Supercomputers Aid Scientists Studying the Smallest Particles in the Universe

Since the 1930s, scientists have been using particle accelerators to gain insights into the structure of matter and the laws of physics that govern our world. Today, these accelerators are some of the most powerful experimental tools available, propelling particles to nearly the speed of light and then colliding them to allow physicists to study the resulting interactions and particles that form.

Many of the largest particle accelerators aim to provide an understanding of hadrons, which are subatomic particles, such as protons or neutrons, made up of two or more particles called quarks. Quarks are among the smallest particles in the universe, and they carry only fractional electric charges. Scientists have a good idea of how quarks make up hadrons, but the properties of individual quarks have been difficult to tease out because quarks cannot be observed outside their respective hadrons. 

Using the Oak Ridge Leadership Computing Facility’s (OLCF’s) IBM AC922 Summit supercomputer, a team of nuclear physicists led by Kostas Orginos at the Thomas Jefferson National Accelerator Facility and William & Mary has developed a promising method for measuring quark interactions in hadrons and has applied this method to simulations using quarks with close-to-physical masses. To complete the simulations, the team used a powerful computational technique called lattice quantum chromodynamics (LQCD) coupled with the computing power of Summit, the nation’s fastest supercomputer. The results were published in Physical Review Letters.

“Typically, scientists have only known a fraction of the energy and momentum of quarks when they’re in a proton,” said Joe Karpie, postdoctoral research scientist at Columbia University and leading author on the paper. “That doesn’t tell them the probability that quark could turn into a different kind of quark or particle. Whereas past calculations relied on artificially large masses to help speed up the calculations, we now have been able to simulate these at very close to physical mass, and we can apply this theoretical knowledge to experimental data to make better predictions about subatomic matter.”

The team’s calculations will complement experiments performed on the US Department of Energy’s (DOE’s) upcoming Electron-Ion Collider (EIC), a particle collider to be built at Brookhaven National Laboratory (BNL) that will provide detailed spatial and momentum 3D maps of how subatomic particles are distributed inside the proton.

Understanding the properties of individual quarks could help scientists predict what will happen when quarks interact with the Higgs boson. The Higgs boson is an elementary particle that is associated with the Higgs field, a field in particle physics theory that gives mass to matter that interacts with it. The method could also be used to help scientists understand phenomena that are governed by the weak force, which is responsible for radioactive decay.

Simulations at the smallest scales

To paint an accurate picture of how quarks operate, scientists typically must average the properties of quarks inside of their respective protons. Using results from collider experiments—such as the ones from the Relativistic Heavy Ion Collider at BNL, the Large Hadron Collider at CERN, or DOE’s upcoming EIC—they can extract a fraction of a quark’s energy and momentum.

But predicting how much quarks interact with particles, such as the Higgs boson, and calculating the full distribution of quark energies and momenta have remained long-standing challenges in particle physics.

To begin tackling this problem, Bálint Joó, who recently joined the Advanced Computing for Nuclear, Particles, & Astrophysics Group at the OLCF, turned to the Chroma software suite for QCD and NVIDIA’s QUDA (for QCD on CUDA) library. LQCD gives scientists the ability to study quarks and gluons—the elementary glue-like particles that hold quarks together—on a computer by representing space-time as a grid or a lattice on which the quark and gluon fields are formulated. Using Chroma and QUDA, Joó generated snapshots of the strong-force field in a cube of space-time, weighting the snapshots to describe what the quarks were doing in the vacuum. Other team members then took these snapshots and simulated what would happen as quarks moved through the strong-force field.

“If you drop a quark into this field, it will propagate similarly to how dropping an electric charge into an electric field causes electricity to propagate through the field,” Joó said.

The difference is that quarks interact with the strong force via strong-force charges known as color charges (purely a naming convention) rather than the two electric ones (positive and negative). Based on the theory, only particles with a combination that forms a “zero” color charge are allowed to exist. Three quark propagators can yield a proton or a neutron, depending on their charges. Likewise, two quark propagators can yield mesons, which are particles that consist of one quark and one anti-quark.

With a grant of computational time from DOE’s Innovative and Novel Computational Impact on Theory and Experiment program, as well as support from the Scientific Discovery through Advanced Computing program and the Exascale Computing Project, the team took the propagator calculations and combined them using Summit to generate final particles from which they could then use to extract results. Summit is located at the OLCF, a DOE Office of Science User Facility located at DOE’s Oak Ridge National Laboratory. 

“We set what are known as the bare quark masses and the quark-gluon coupling in our simulations,” Joó said. “The actual quark masses, which arise from these bare values, need to be computed from the simulations—for example, by comparing the values of some computed particles to their real-world counterparts, which are experimentally known.”

Drawing from physical experiments, the team knew that the lightest physical particles they were simulating—called the pi mesons, or pions—should have a mass of around 140 megaelectron volts (MeV). The team’s calculations ranged from 358 MeV down to 172 MeV, close to the experimental mass of pions.

The simulations required the power of Summit because of the number of vacuum snapshots the team had to generate and the number of quark propagators that needed to be calculated on them. To estimate the results at the physical quark mass, calculations needed to be performed at three different quark masses and extrapolated to the physical one. In total, the team used more than 1,000 snapshots over three different quark masses in cubes with lattices ranging from 32³ to 64³ points in space.

“The closer the masses of the quarks in the simulation are to reality, the more difficult the simulation,” Karpie said. “The lighter the quarks are, the more iterations are required in our solvers, so getting to the physical quark masses has been a major challenge in QCD.”

Algorithmic advances bring new opportunities

Joó, who has been using the Chroma code on OLCF systems since 2007, said that improvements in algorithms over the years have contributed to the ability to run simulations at the physical mass.

“Algorithmic improvements like multigrid solvers and their implementations in efficient software libraries, such as QUDA, combined with hardware that can execute them have made these kinds of simulations possible,” Joó said.

Although Chroma is his bread-and-butter code, Joó said advances in code development will continue to provide opportunities to target new challenge problems in particle physics.

“Despite having worked with this same code all these years, new things still happen under the hood,” Joó said. “There will always be new challenges because there will always be new machines, new GPUs, and new methods that we will be able to take advantage of.”

In future studies, the team plans to explore gluons and obtain a full 3D image of the proton with its various components.

The team members in this work are members of USQCD, a collaboration of US scientists developing and using large-scale computers for calculations in lattice QCD.

Related Publication: Bálint Joó, Joseph Karpie, Kostas Orginos, Anatoly V. Radyushkin, David G. Richards, and Savvas Zafeiropoulos. “Parton Distribution Functions from Ioffe Time Pseudodistributions from Lattice Calculations: Approaching the Physical Point,” Physical Review Letters 125 (2020): 232003, doi:10.1103/PhysRevLett.125.232003.