Projects in Scientific Computing

Download the PDF version of this article as it appeared in Projects in Scientific Computing, 2002.


"Three quarks for Muster Mark!
Sure he hasn't got much of a bark
and sure any he has it's all beside the mark."

James Joyce, Finnegan's Wake

Who says physicists aren't fun? When we want to talk about the fundamental particles of matter, the rock-bottom, indivisibly smallest of the small things deep within the nucleus of an atom, we can be grateful that the person who thought them up read James Joyce and had a sense of humor.

In 1963, theoretical physicist Murray Gell-Mann needed a name for his idea for the building blocks of protons, neutrons and related subatomic particles, till then thought to be indivisible. He settled on quark, a word that combines a dog's bark with a sea gull's squawk and that in German means curd, the fundamental constituent of cheese. Gell-Mann's quarks, which originated as a mathematical glimmer in his mind, have since been substantiated in many experiments, though no one — for reasons inherent in the nature of quarks — has yet tasted, touched, seen or in any way directly observed one.

Gell-Mann won a 1969 Nobel Prize, and by the early 1970s, quarks were key ingredients in a powerful theory, called quantum chromodynamics or QCD, that describes the strongest force in nature — the interactions that hold together the nucleus of an atom. QCD is a major component of modern physics' integrated theory of the fundamental particles and their interactions, called The Standard Model.

"The Standard Model," says Indiana University physicist Steven Gottlieb, "has been enormously successful, having passed all the experimental tests to which it's been put. But what we know about it is incomplete, because it's been difficult to extract many of the most interesting predictions of QCD."

Enter supercomputing. Though frightfully challenging and demanding the most powerful systems that can be built, computational simulations of QCD are crucial to expanding what's known in high energy and nuclear physics.

structure of magainin

Members of the MILC group, colleagues and canine friends at a recent Lattice QCD conference in Salt Lake City (l to r): Carleton DeTar, Eric Gregory, Tom Burch, James Osborn, Roberta Toussaint, Doug Toussaint, Steve Gottlieb, Phillip Toussaint.

Gottlieb began doing QCD computations in the early 1980s, and since the 1990s he's been part of a nationwide team of theoretical physicists who collaborate to gain maximum advantage from the available computing time. They call themselves the MILC collaboration, an acronym, MImd Lattice Computation, that points both to scientific computing (multiple instruction, multiple data — mimd — the technology of massively parallel processing) and to the computational approach — the "lattice" — that makes it feasible to solve QCD equations.

LeMieux, PSC's terascale system, has been a major boost to their work. Beginning in early 2001 with the TCSini prototype and since October, when grownup LeMieux came online, the MILC team has milked every drop of computing available to it from this system. The result is their most realistic QCD calculations yet. "We've made exceptional progress," says Gottlieb, "that wouldn't have been possible without this system."

May the Forces Be With You: The Standard Model & CKM Matrix Elements

The Standard Model is a pedestrian name for one of the more glorious achievements of science. It's a tidy summing up of much of the knowledge that has blossomed, if not exploded, from the 20th century revolutions of quantum theory and relativity. It links the tiniest particles inside an atom with the big bang and what we know about evolution of the cosmos.

Four forces — that's all you need to know. From the standpoint of modern physics, everything that happens is at root level a result of gravity, electromagnetism, the strong force and the weak force. For The Standard Model, gravity is the stepchild that doesn't quite fit. The weakest of the four — weaker even than the weak force — gravity affects us only when matter aggregates into huge masses, such as planets and stars.

The Standard Model's fame lies in bringing the other three forces under one umbrella. It includes a unified theory of electromagnetism and the weak force (electroweak theory) and QCD, the theory of the strong force. In other words, if a dream of modern physics is (and it is) a theory of everything (TOE), The Standard Model is close to three-fourths of a TOE.

The Standard Model sums up what we've learned about fundamental particles and how they interact at the extremes, such as when atoms are smashed in a collider and "decay" into other particles. Most fundamental particles exist only for fleeting instants as traces from these high-energy events, in which scientists force them out from hiding.

As successful as The Standard Model has been, physicists know it's a plateau in the progress of science — a grand edifice that with every piece of knowledge added reveals another question to be resolved. QCD calculations are the only way "to confront experiment with theory," says PSC physicist and co-scientific director Ralph Roskies — to compute quantities needed to answer some of the questions. While we spend hundreds of millions annually on powerful accelerator experiments, QCD calculations — which aren't backed with this level of resources — are an equal partner in the intellectual effort.

"We need simulations to determine a number of the basic parameters of The Standard Model," says Gottlieb, "for quantitative understanding of physical phenomena controlled by the strong interactions, and to do precision tests of the Standard Model."

Among the payoffs ahead — awaiting accurate QCD calculations to interact with experimental data — are quantities called CKM matrix elements. Named for three physicists, Cabibbo, Kobayashi and Maskawa, CKM matrix elements are of great interest to contemporary physics. They are fundamental parameters of The Standard Model, similar to Planck's constant at the core of quantum theory. They define the intersection between the strong and weak force in events that involve both. Some of the CKM elements are known. Others are more intractable. Pinning them down could lead the way to deeper levels of understanding in particle physics.

Hand drawn diagram of particle interaction.

Weak Decay
Called a Feynman diagram, after Nobel physicist Richard Feynman who originated them, this drawing by Steve Gottlieb starts on the left with a B meson (B+). The strong force confines an up quark (u) and bottom anti-quark () inside the B meson, where they interact until they annihilate. "For a moment," says Gottlieb, "there's a W boson (W+), carrier of the weak force. Then that W boson, because it's a virtual particle, decays to an anti-electron, also called a positron (e+), and an electron neutrino (_e)."

Quarks, Mesons & Weak Decay

According to QCD, quarks come in six "flavors" — up, down, strange, charm, bottom and top (in ascending order by mass) — that mix in different combinations to form many varied species of the subatomic particle zoo. The big particles of the nucleus, protons and neutrons, are bundles of three quarks. Two ups and a down make a proton, an up and two downs a neutron.

From the mid-1960s forward, accelerator experiments have set out to break quarks free from the larger nuclear particles they inhabit. No luck. The problem, as it's been theorized within QCD, is that when you add great amounts of energy to a proton — trying to smash it in an accelerator — rather than freeing a quark, the added energy becomes new matter. Out of thin air, another particle appears. The particle that forms is a meson — a quark paired with an anti-quark.

There are many mesons, and one of them in particular, the B meson — B for the bottom anti-quark (paired with an up or down flavored quark) - is the subject of a tantalizing question, one the MILC group is aiming at with LeMieux. The question — "weak decay" — is one of the places where QCD calculations are crucial to determining CKM matrix elements.

The weak force — in some ways the most mysterious of the four forces - is responsible for nuclear decay, of which the best known example is radioactivity. Because of the weak interaction, the vast majority of fundamental particles when left to themselves disintegrate to one of only a few apparently stable particles — prominently including protons, electrons and neutrinos.

With a B meson, a "leptonic" weak decay happens when the heavy anti-quark and light quark annihilate each other due to the weak interaction. (Bottom, the next to heaviest quark, is 1,000 times heavier than up, the lightest.) What comes out of this decay, roughly analogous to sex change, is no longer a nuclear particle but a pair of non-nuclear particles. Called leptons (Greek for light and swift), the most prevalent are electrons and neutrinos.


Gottlieb and his colleagues and many others want to accurately describe how the weak interaction contributes to this leptonic decay. But the strong interaction gets in the way. The quark and anti-quark stay inside the meson, interacting through the strong force before they annihilate. "If we don't understand how they interact strongly," says Gottlieb, "we can't get the detailed probability of how they'll decay."

Major accelerator experiments now underway, one at Stanford and another in Japan, are focused on producing B mesons and analyzing their decay. At the same time, with big help from LeMieux, the MILC group is zeroing on a quantity — called the B meson decay constant — that's needed, along with data from the experiments, to arrive at the CKM matrix elements that define this decay.

Building a Better Lattice

The MILC QCD Collaboration
Claude Bernard, Washington University
Tom DeGrand, University of Colorado
Carleton DeTar, James Osborn, University of Utah
Steven Gottlieb, Indiana University
Urs M. Heller, Florida State University
James Hetrick, University of the Pacific
Robert Sugar, University of California, Santa Barbara
Tom Burch, Eric Gregory, Doug Toussaint, University of Arizona

The prerequisite to calculating the B meson decay constant is realistic QCD lattices, a scaffold-like framework that divides the continuum of space-time into discrete regions in which to compute physical quantities. "We're generating the most realistic set of lattices we've ever produced," says Gottlieb of the past year's work with LeMieux. The major part of the MILC effort has been to generate the best lattices possible with available resources. With over a million single-processor hours of LeMieux (nearly 3,000 hours on 512 processors), their new lattices are more finely resolved than ever (grid spacing is .09 femtometers, a million-billionth of a meter).

Significantly, the LeMieux lattices also reflect a more realistic representation of dynamical quarks — the sea-like background of quark, anti-quark pairs that affect QCD physics even though they aren't confined in larger particles. Prior QCD lattices have included only up and down dynamical quarks. The new lattices, for the first time in QCD, include dynamical strange quarks. Preliminary analysis indicates that these quarks influence the value of the decay constant.

With the new lattices and more computations, the MILC group is progressively closing in on the elusive pieces of knowledge it seeks. As powerful an explanatory work as it is, The Standard Model is replete with mystery. "Why are the quark masses what they are?," asks Gottlieb. "The top quark is 175 times the mass of the proton. It's amazing. Quarks and leptons come in pairs of three generations. Why?"

The CKM matrix elements represent a key that may unlock the next layer of mystery. "A more basic, deeper theory," says Gottlieb, "would have to make an accurate prediction of the CKM matrix elements. Until we can accurately extract these parameters, there will be uncertainty when people propose more fundamental theories. We all think there may be something beyond The Standard Model, something we don't yet understand."

The MILC Collaboration

Terascale Computing System

MILC code.

Related Material on the Web:


Michael Schneider, Pittsburgh Supercomputing Center

Web Design:
Sean Fulton, Pittsburgh Supercomputing Center

Photo Credits:
Laurel Casjens.

Revised: November 15, 2002