Colin Morningstar (center), graduate students Chik Him Wong (left) and John Bulava, Carnegie Mellon University
“Nothing exists except atoms and empty space; everything else is opinion,” wrote Democritus around 400 B.C. His notion of a single fundamental unit of matter, an atom, lasted more than two thousand years, until during the past century we learned that what we call the atom is itself a world of smaller things — a nucleus of protons and neutrons surrounded by clouds of electrons.
By the 1960s, it became apparent that protons and neutrons are themselves made from smaller things yet. In 1963 Murray Gell-Mann dubbed them quarks, and by the 1970s a powerful theory called quantum chromodynamics (QCD) arose to describe how the strongest force known — the “strong interaction” — acts via things called gluons to bundle three quarks into protons and neutrons.
From collider experiments over many years, scientists learned that protons and neutrons fragment and recombine into a vast array of related particles. As a family, they're called hadrons — from Greek, hadros, for stout, thick — with the common denominator that they're made from quarks and gluons interacting via the strong force. Unlike a proton — the stable “ground state” hadron — most hadrons exist only at much higher energies for infinitesimally brief flickers of time. To identify these higher-lying states — known as “excited states” — and find them in collider experiments is a major challenge of physics, involving thousands of scientists internationally.
Protons would fly apart except for the strong nuclear force, which is carried by gluons (force lines between quarks). When gluons clump together they form a “glueball.” Modern conception of the proton includes more than the three “valence” quarks — a down (d) and two up (u) quarks — which account for only about 2 percent of the proton’s mass. The rest comes from a “sea” of virtual quarks and glueballs. Even outside the bounds of the proton, virtual particles spring into and out of existence within the teeming vacuum described by QCD. (Rendering courtesy of Alex Dzierba, Curtis Meyer and Eric Swanson)
“What are the hadrons? What’s the spectrum of QCD?” says Colin Morningstar of Carnegie Mellon University. “What hadrons come from the quark and gluon fields? What particles can we observe in the lab, and what are their properties?” With these fundamental questions as impetus, Morningstar and his colleagues in a team called the Hadron Spectrum Collaboration (HSC) last year achieved a breakthrough. Using resources at several TeraGrid sites — TACC, NICS and PSC's BigBen — and at DOE, they applied a sophisticated, computationally demanding approach called lattice QCD. Doing what no one else had previously been able to do, they successfully calculated the masses of an excited-state spectrum of hadrons.
Lattice QCD computations are essential as a means to reconcile QCD theory with experiments and extract insights into the physics of hadrons, and the HSC team’s findings — reported in Physical Review D (January 2009) — will guide accelerator experiments, especially at the U.S. DOE Thomas Jefferson National Accelerator Facility in Virginia. “For the experimentalists,” says Morningstar, “the effects of these excited states are hidden inside scattering cross-sections, and it’s like looking for a needle in a haystack. With our findings, they can do a much better job of figuring out what’s there.”
The HSC team’s success built on decades of computational work in QCD and relied on several innovations of their own. Developed by Nobel-prize winner Kenneth Wilson in the 1970s, lattice QCD makes it possible to solve the complex equations of QCD, using the most advanced supercomputers, by imposing a gridlike structure on the space and time that quarks inhabit.
The Spectrum of Proton and Neutron Excited States
This mass plot (vertical axis) with different symmetry channels (horizontal axis) represents the spectrum of excited states for the proton or neutron (lower left horizontal bar). Each colored box represents a different nucleon — states of the proton or neutron, with the height of the box indicating the amount of statistical uncertainty associated with it. (Colors make it possible to distinguish different states and have no physical meaning.) Only the proton/neutron and the left-most green box had been identified in QCD calculations before this work. The G2g channel shows spin 5/2 particles, never before calculated in lattice QCD. “It’s the pattern that represents the physics,” says Morningstar, “and this pattern hadn’t been seen before in lattice QCD calculations.”
Within this lattice, quarks exist at the cross points and forces act along the lines between cross points. For their excited-state computation, Morningstar and colleagues created a lattice of 24 points in each of the three spatial dimensions and 64 points in time. A proton has a diameter of approximately one femtometer (one-quadrillionth of a meter, 10-15 m), and it’s been estimated that a quark is at least 1,000 times smaller than a proton, if it can be measured at all. To model a system this small, the HSC lattice had spacing of 0.11 femtometers in space and 0.033 femtometers in the time dimension.
Using from 2,000 to 4,000 processors on several different systems, including BigBen, they relied on an approach — called Hybrid Monte Carlo (HMC) — implemented through a software package (called Chroma) developed by Morningstar’s HSC colleagues and other lattice QCD theorists. The Monte Carlo method, as the name implies, uses random numbers as inputs and from them estimates complicated integrals over the gluon field configurations. “HMC is a sophisticated way of proposing new gluon fields,” says Morningstar, “with the appropriate probability distribution, taking into account the effects of the propagating quarks.”
HMC is the first step in the computation, producing an ensemble of possible configurations of the quark-gluon system. For the next step, the researchers use hadron “operators” — mathematical formulations of the quark and gluon fields. They apply these operators to the configurations generated in the first step. Finally, they fit the results to an exponential equation, which yields a hadron’s energy, hence its mass — due to the equivalence of mass and energy.
An essential benefit of HMC is that the researchers didn’t have to construct a single, correct operator, but could use a set of operators to arrive at the proton’s excited states. “We don’t know exactly what the proton is,” says Morningstar, “and we can’t make a proton operator, but we can make an operator that has all the same symmetry properties as the proton. So it will create the proton, but it will also create other particles that have the same symmetry properties. They are the higher lying ones, the excited states.”
The HSC team’s computational advance that made their physics breakthrough possible was their ability to systematically design operators to focus on particular properties. They used a technique called “smearing” to reduce unwanted “noise” — inherent in the statistically based computations. “A lot of work goes into operator construction,” says Morningstar. “There’s a myriad of excited states, but if you design your operator in the right way, you can kill off couplings to most of the unwanted states.”
Smearing involves averaging the field over a larger volume of space, but in such a way that symmetries are left intact. If the field had rotational symmetry before the smearing — meaning it looks the same when you rotate it, say, 90° — it has to retain this symmetry after it is smeared. “It has been known for some time — but we really exploited it here,” says Morningstar, “that if you make your operators out of smeared quark and gluon fields, you get a signal that plateaus to the state you want much faster.”
Besides speed, smearing has the benefit of eliminating signals from most of the myriad interfering, high-energy states. If the quark or gluon field used in the operator is a small, point-like field, it can “couple” to these interfering states. When the field is smeared-out instead of point-like, the interfering state disappears. “The goal of operator construction,” says Morningstar, “is to get it to excite the state you want and nothing else; smearing is a crucial part of that.”
The HSC group had another trick up its sleeve. They used higher-than-realistic quark masses and smaller-than-normal volumes to simplify calculations that were, nevertheless, enormously demanding. “This was a first demonstration of the computational technology to show you could do this,” says Morningstar. “A lot of people in our community didn’t think it was possible. We did the simpler calculation first, and now the next step is to go to larger volumes, and to get down to physical quark masses.”
Their results correlate well with observed findings, and they expect further computations to improve them. Since its formulation about 30 years ago, lattice QCD has pushed the limits of what’s possible with high-performance computing. The HSC group, which uses all the computing it can get, is already hatching plans for how to tackle lattice QCD with exascale systems. “We want to be able to get out what the physical predictions of this theory are,” says Morningstar, “and we just keep applying pressure to it. It’s a tough nut to crack.”