Blacklight Goes to Work

With a $2.8M award from the National Science Foundation, PSC introduced the world's largest shared-memory supercomputer.

Researchers are making productive use of Blacklight. This new system, which PSC acquired in July 2010, with help from a $2.8 million award from the National Science Foundation, has opened new capability for U.S. scientists and engineers. With 512 eight-core Intel Xeon 7500 (Nehalem) processors (4,096 cores) and 32 terabytes of memory, Blacklight is partitioned into two connected 16-terabyte coherent shared-memory systems — the two largest shared-memory systems in the world.

PHOTO: Blacklight

In computer terms, "shared memory" means a system's memory can be directly accessed from all of its processors, as opposed to distributed memory (in which each processor's memory is directly accessed only by that processor). Because all processors share a single view of data, a shared memory system is, relatively speaking, easy to program and use.

“For many research communities — including data analysis and many areas of computer science," said PSC scientific directors Michael Levine and Ralph Roskies in October 2010, as Blacklight became a production resource, "Blacklight opens the door to high-performance computation and thereby expands the abilities of scientists to ask and answer questions." As described in this publication, Blacklight has already enabled advances in nanomaterials (p. 8), genomics (p. 28), natural language processing (p. 32), astrophysics (p. 40), geophysics (p. 44), machine learning, (p. 46) and climate modeling (p. 47).


Jim Kasdorf, PSC director of special projects

Blacklight Memory Advantage Program

To help researchers take advantage of Blacklight, PSC provides a Memory Advantage Program to develop applications that can effectively use Blacklight's shared-memory capabilities. These include rapid expression of algorithms — such as graph-theoretical software, for which distributed memory often presents obstacles, and interactive analysis of large data sets, which often can be loaded in their entirety into Blacklight's shared memory. For such projects, a PSC consultant can provide advice on debugging and performance-analysis tools and procedures, and other fixes and optimizations. Interested researchers may contact:


As a leading partner in XSEDE, the most powerful collection of integrated digital resources and services in the world, PSC helps to shape the vision and progress of U.S. science and engineering.


Through XSEDE, the Extreme Science and Engineering Discovery Environment, the NSF cyberinfrastructure program that launched this year, PSC extends its active role in the development of national cyberinfrastructure. XSEDE replaces and expands on the TeraGrid, the predecessor NSF program that began more than a decade ago. More than 10,000 scientists used TeraGrid to complete thousands of research projects. Similar work — only in more detail and in a broader range of fields — continues with XSEDE.

PSC-scientific co-director, Ralph Roskies is a co-principal investigator of XSEDE and co-leads its Extended Collaborative Support Services (ECSS). "ECSS staff work both with user groups in fields familiar with high-performance computing," says Roskies "and with the XSEDE outreach team to reach user groups, communities and digital services that are new to HPC."

Other PSC staff lead many areas of the comprehensive XSEDE program. Janet Brown, who manages PSC's network research, leads the XSEDE Systems and Software Engineering team that oversees the software environment that integrates resources among many providers. As manager of XSEDE Outreach Services, PSC manager of education, outreach and training Laura McGinnis leads programs that help to prepare the next generation of computational scientists.

PSC's security officer, Jim Marsteller, is the Incident Response Lead for XSEDE. Wendy Huntoon, PSC director of networking, is XSEDE Networking Lead. Ken Hackworth, PSC's user relations coordinator, leads the XSEDE allocations process by which research proposals are reviewed and evaluated to receive grants of computational time on XSEDE resources. PSC scientist Sergiu Sanielevici, director of scientific applications and user support for PSC, leads the Novel and Innovative Projects area of XSEDE's ECSS effort, which focuses on development of projects in fields or from institutions and communities that can exploit advanced computing but haven't traditionally used it.


PSC's directors (l to r), who oversee day-to-day PSC operations and help to coordinate PSC's role in XSEDE: Cheryl Begandy, director, education, outreach & training; Bob Stock, PSC associate director; David Kapcin, director of financial affairs; Sergiu Sanielevici director, scientific applications & user support; David Moses, executive director; Wendy Huntoon, director of networking. Not pictured: Nick Nystrom, director, strategic applications; J.Ray Scott, director, systems & operations.