Research Notes & Highlights, 2005
- The National Resource for Biomedical Supercomputing
- Networking the Future
Established in 1987 as one of the very first biomedical supercomputing programs in the country and renamed this year, the National Resource for Biomedical Supercomputing (NRBSC) pursues leading-edge research in the life sciences while exploiting high-performance computing and applying PSC expertise in computational science to collaborative biomedical research nationwide.
With support from NIH’s National Center for Research Resources, NRBSC also conducts workshops and courses to train scientists in the use of high-performance computing for biomedical research, in areas such as spatially realistic cell modeling, volumetric data visualization and analysis, protein and DNA structure, genome sequence analysis and biological fluid dynamics. Since 1987, PSC biomedical training has reached more than 3,200 researchers, and PSC computational resources have supported more than 1,000 biomedical research projects involving more than 2,500 researchers at 218 research institutions in 48 states. Among these are several projects featured in this booklet (Fire in the Brain, All in Your Brain).
With colleagues at the Salk Institute, scientists at PSC’s Center for Quantitative Biological Simulation and NRBSC are developing MCell and DReAMM, software for simulation and visualization of physiologically realistic cellular models (see Fire in the Brain). New versions of both programs include major new features. MCell can now simulate diffusion and reaction of multiple chemical species in solution, enabling much broader studies of metabolic, signaling and regulatory networks. DReAMM can visualize complex MCell models containing thousands of different objects and, as shown here, can also represent volumetric data. This example shows a finite-element simulation of electric field strength. Future versions of MCell will integrate Monte Carlo and finite-element algorithms for quantitative multi-scale cellular simulations.
With support from the National Library of Medicine, NRBSC scientists Arthur Wetzel, Stuart Pomerantz, Demian Nave and David Deerfield have developed a new version of the PSC Volume Browser, PSC-VB II. The original PSC-VB, released in 2002, operated with large static anatomical volumes such as the NLM Visible Human datasets. PSC-VB-II has many additional features needed for “gene knockout” research or, as shown here, studies of 3D motions in living animals. The image — from a micro-CAT scan analysis by NRBSC research partners at the Duke University Center for In Vivo Microscopy — shows a mouse heart. Producing three gigabytes of volume data, the scan captured eleven steps in the heartbeat cycle. Using PSC-VB II, researchers track the region of interest (blue outlined in green) to study volume and other changes in the left ventricle.
Genes are blueprints for proteins, and the flood of sequence data from genomic research challenges scientists to take the next step and use this massive data to deduce the 3D structures of proteins. NRBSC scientist Troy Wymore and University of Pittsburgh student Adam Marko participated in CASP6, an international experiment by which molecular biologists assess techniques to do this. Held every two years, CASP sets up blind tests with sequences for structures known but not published. In 2004, 166 research groups took part. The PSC group developed a method that combines comparative modeling, statistical potentials and replica-exchange simulations, and they achieved several high-scoring results. For a protein from a small flowering plant named arabidopsis, their proposed structure was the fifth-best worldwide.
Species evolve over time because their genes change. Some patches of DNA copy perfectly and others drop out or fail to replicate exactly, leading to changes in the molecular processes of the species. NRBSC biological chemist Hugh Nicholas specializes in computational methods to identify and track these changes and thereby to reconstruct the history of genes. In recent work, Nicholas is collaborating with Carnegie Mellon biologist Dannie Durand to trace the family tree for a large family of enzymes, called glutathione S-transferases, that help rid the body of toxins such as environmental pollutants. Better knowledge of this enzyme family could help in developing drugs to improve bioremediation efforts and chemotherapy treatment.
PGENESIS is a parallelized version of widely used software for modeling biological neural systems such as the brain. Originally developed at PSC by biomedical researchers Nigel Goddard and Greg Hood, PGENESIS extends software called GENESIS to parallel systems and makes it possible to do multiple simulations and “parameter searches” — identify and zero-in on the most interesting among a set of simulations while they are running. This year, PSC released a new version of PGENESIS that integrates support for interprocessor communications and facilitates its use on workstation clusters and massively parallel systems such as LeMieux and Big Ben.
The PSC operational management team: (front, l to r) David Kapcin, manager, financial affairs; Nick Nystrom, director, strategic applications; Rich Raymond, manager, user support; Sergiu Sanielevici, director, scientific applications and user support; Gwendolyn Huntoon, director, networking; (back, l to r) John Kochmar, HPC facilities manager, systems and operations; Joel Stiles, director, Center for Quantitative Biological Simulation, National Resource for Biological Supercomputing; J. Ray Scott, director, systems and operations; Bob Stock, associate director. Not in photo: Janet Brown, manager, networking; David Deerfield, director, Biomedical Initiative.
PSC’s Advanced Networking group is one of the leading resources in the world for knowledge about networking. Through 3ROX (Three Rivers Optical Exchange), a high-speed network hub, they provide high-performance networking for research and education. Their research on network performance and analysis — in previous projects such as Web100 and current work with HPN-SSH and pathdiag — has created valuable tools for improving network performance nationally.
|3ROX Members||Network Connections|
|Carnegie mellon University, University of Pittsburgh, Pennsylvania State University, West Virginia University, Pittsburgh Public Schools||
National Research NetworksAbilene—2.4 Gbps, TeraGrid Extensible Backplane Network—30 Gbps.
|The National Energy Technology Laboratory, NASA Independant Verification and Validation Facility||
National Commodity Internet NetworksAT&T Worldnet—622 mbps, Global Crossing—622 mbps
|Comcast, Westinghouse Electric Co.||
Local PeersPITX (Pittsburgh local exchange network), Comcast, TelCove
|Computer Emergency Response Team|
Through 3ROX, a high-speed network hub that serves Carnegie Mellon, Penn State, the University of Pittsburgh, West Virginia University and the Pittsburgh Public Schools, PSC provides advanced network resources for education and research. 3ROX connects the universities and PPS to Abilene a high-performance network linking more than 250 U.S. universities and research organizations.
This year a new high-speed fiber linking TelCove, a major provider of telecommunication services, with 3ROX improved efficiency and speed of access for TelCove users in the Pittsburgh region. In July, 3ROX turned on the next-generation Internet protocol, called IPv6, with the commodity network Global Crossing, thereby becoming the first network hub to implement IPv6 on both research and commodity networks, a change which improves performance for 3ROX members connecting to IPv6 sites.
Wendy Huntoon, PSC director of networking
In September 2005, PSC director of networking Wendy Huntoon was appointed director of operations for National LambdaRail, a major initiative of U.S. research universities and the private sector to provide infrastructure for research in networking technologies.
“The members of NLR will benefit immensely from Wendy’s vast experience at the frontier of high-performance research and education networking,” said Tom West, CEO of NLR. “With her longstanding leadership in the advanced networking community, she brings an extremely valuable set of skills to the task of finalizing the buildout of the NLR infrastructure.”
NLR is deploying a nationwide optical network that provides up to 40 simultaneous light wavelengths, each capable of transmitting 10 gigabits per second. This infrastructure simultaneously supports cutting-edge network research and ambitious scientific projects that require guaranteed networking reliability and performance.
“It’s a privilege to be able to help put this infrastructure in the hands of researchers who will create the next generation of networking technologies,” said Huntoon. “As we improve the performance of our networks and experiment with new networking techniques, we’ll be enabling a new breed of high-performance and network applications.”
Unprecedented Connectivity: Last November at Supercomputing ‘05 in Pittsburgh, PSC’s network group accomplished a “first” in high-performance networking. Using two Cisco CRS-1 routing systems fitted with OC-768 interfaces, they demonstrated 40-Gigabit per second transmission over a single “lambda” — a light wave — in actual application processing.
The demonstration relied on PSC-developed protocol that coupled an application running on LeMieux to the showfloor network at SC ‘05. “This is the next step forward in networking,” said PSC network director Wendy Huntoon. “We demonstrated PSC’s ability to fully utilize this bandwidth with real application data.”
Eliminating a Network Bottleneck: SSH (Secure Shell) is a much used security-enhanced protocol for network communications, freely available for many operating systems and network products from Apple, Sun, HP, IBM, Cisco and Juniper. SSH, however, often limits the network performance that’s potentially available on high-speed networks. Members of the PSC network group this year created a software patch to overcome this bottleneck. In many circumstances, this patch, called HPN-SSH, can give more than a 10-fold speedup in performance.
NASA has implemented this patch as part of their Solar Dynamics Observatory mission. “This is helping us tremendously,” said NASA scientist Deepak Paul, “because it allows us to keep our real-time requirements and our security requirements. Previously, we could support only one or the other.”
Improved Network Debugging: Network performance debugging is difficult because most flaws show up with the same symptom: slowed performance. The nature of the problem causing slowed performance is often “masked” by self-correcting routines built into network protocol, and to identify the precise cause of the slowdown poses a difficult challenge. To deal with this, PSC engineers in collaboration with the National Center for Atmospheric Research (NCAR) developed innovative software — called pathdiag — that clearly and accurately diagnoses flaws and prescribes corrective action. This technology, which draws on tools PSC helped to develop in a project called Web100, is now in use at NCAR, the University of Michigan, Duke University and the Lawrence Berkeley National Laboratory.