NSF/PACI Cluster Computers

User and Developer Notes
Cray XT3 (bigben)

The Cray XT3 consists of two rows of 11 cabinets. Each cabinet has three shelves. Each shelf includes 8 modules. Service modules have two sockets (4 cores); compute modules have four (8 cores.) Approximately 2068 dual-core compute nodes are generally available.

All processors are 2.6 GHz AMD Opterons with a 1 MByte cache.

The network is a Cray proprietary design called SeaStar. The SeaStar chips are coupled to the processors via AMD's HyperTransport channel. Network topology is torispherical. API is Sandia Portals.

Service nodes run SuSE Linux; compute nodes run microkernel code call Catamount. Compiling systems are by Portland Group. The batch system is PBS.
HP/Compaq ES45 (lemieux)

This system was retired on December 22, 2006.

The HP/Compaq ES45 cluster at Pittsburgh Supercomputing Center is comprised of 764 ES45 Alphaserver nodes of which two are designated for interactive logins.

Full machine jobs are supported by the queuing system. The queue is drained every afternoon in preparation for the gating of large jobs at 8 am Eastern.

All nodes feature four 1000-MHz Alpha EV6.8CB (21264C) processors and 4 GBytes of memory.

All nodes are currently running the Tru64 UNIX V5.1A AlphaServer SC TS2.5 operating system. The batch system is PBS.

The interconnect fabric is based on Quadrics' QsNet product (340 MBytes/s peak and 210 MBytes/s sustained x 2 rails).
HP GS1280 (jonas and rachel)

There are two Hewlett-Packard GS1280 SMP clusters at Pittsburgh Supercomputing Center. Each is comprised of a DS20E login node and two GS1280 7/1150 compute nodes. These systems are essentially identical.

A significant portion of the funding for jonas was provided by NIH, while rachel is an NSF machine. Allocations follow these lines.

Each DS20E login node features two (2) EV6.7 (21264A) processors operating at 667 MHz, and 1 GByte of memory. These processors have an 8 MByte cache.

Each GS1280 7/1150 compute node has sixty-four (64) 1150 MHz EV7 (21364) processors and 256 GBytes of memory. These processors have a 1.75 MByte cache.

All nodes are currently running the Tru64 UNIX V5.1B operating system. The batch system is PBS.
Dell/Intel Xeon Cluster (tungsten)

The Dell PowerEdge Xeon cluster at the National Center for Supercomputing Applications is comprised of 1450 Dell PowerEdge 1750 servers of 1280 are configured as compute nodes. Of the remaining 170 nodes, 104 are Lustre fileservers and ??? are designated login nodes.

Each node features two 3.06-GHz Intel Xeon processors with a 512 MByte full-speed L2 cache. All nodes feature 3 GBytes ECC DDR SDRAM memory.

All machines are running Red Hat 9.0 Linux (2.4.20). The batch system is LSF.

The primary interconnect fabric is based on Gigabit Ethernet. Documentation indicates that Myricom's Myrinet 2000 product is currently unavailable.
HP/Compaq ES40 (ben)

This system is now available to Pennsylvania corporate and academic users.

The HP/Compaq ES40 cluster at Pittsburgh Supercomputing Center is comprised of 64 ES40 Alphaserver nodes and one DS20 login node.

Each compute node features four 667-MHz Alpha EV6.7 (21264A) processors and a 4 GByte memory space. The login node features two 500-MHz Alpha EV6 (21264) processors and 2 GBytes of memory.

All nodes are currently running the Tru64 UNIX V5.1B AlphaServer SC V2.5 operating system. The batch system is PBS.

The interconnect fabric is based on Quadrics' QsNet product (340 MBytes/s peak and 210 MBytes/s sustained x 2 rails).
IBM pSeries 690 (copper)

The IBM p690-Series cluster at the National Center for Supercomputing Applications is comprised of one 16-processor login node and eleven 32-processor compute nodes.

All systems feature 1300-MHz Power4 processors which are capable of delivering 4 results per cycle. Each CPU chip includes two processors. Multichip modules (MCM) are constructed of four chips, or 8 processors.

The memory hierarchy includes three cache levels. Each processor has its own L1 cache (32 KBytes). Each chip has one L2 cache (1440 KBytes), and each multi-chip module has one L3 cache (128 MBytes).

The login node has 32 GBytes of memory and 1 TByte of disk. Four compute nodes have 256 GBytes of memory and 4 TBytes disk; the others have 64 GBytes and 1 TByte.

All nodes are currently running AIX 5.1. The batch system is LoadLeveler.
IBM/Intel IA-32 (platinum)

This system was retired August 30, 2004.

The IBM IA-32 cluster at the National Center for Supercomputing Applications is comprised of 484 IBM eServer x330 thin servers of which four are designated access nodes, and 32 x340 storage nodes (516 total nodes).

All nodes feature two 1000-MHz Intel Pentium III processors. Login and storage nodes have 2 GBytes of memory; compute nodes have 1.5 GBytes.

All machines are running Red Hat 7.2 Linux and Linux 2.4.9-31 SMP kernel with large file support. The batch system is PBS.

The primary interconnect fabric is based on Myricom's Myrinet 2000 product (125 MByte/s).
IBM/Intel IA-64 (titan)

This system was retired July 19, 2004.

The IBM IA-64 cluster at the National Center for Supercomputing Applications is comprised of 130 IBM IntelliStation Z Pro 6894 servers of which two are designated login nodes, and 4 IBM eServer x330 storage nodes.

Each node features two 800-MHz Intel Itanium processors with a 4 MByte full-speed L3 cache. Login and compute nodes feature 2 GBytes of memory; storage nodes have 1.5 GBytes.

All machines are running Red Hat 7.1 Linux for Itanium (2.4.16). The batch system is PBS.

The primary interconnect fabric is based on Myricom's Myrinet 2000 product (240 MByte/s).