High Performance Computing Cluster (HPCC) Update
Fall 2002

**********************************************
HPCC Update
**********************************************

The lease of the 32 processor SGI Origin 2000 (medusa.hpcc.nd.edu) has expired and the machine will be permanently removed on Monday September 9th. Some of the memory/disk which doesn't need to be returned will be moved to the 8 processor SGI Origin 2000 (poseidon.hpcc.nd.edu).

This summer the HPCC staff has been working with IBM personnel on the addition of an IBM 1300 Linux cluster consisting of 32 IBM x330 dual processor servers. Along with the Linux cluster a major upgrade to the HPCC infrastructure includes the addition of an IBM FAStT500 storage server. This storage server is a SAN which provides 1.7 Terabytes (usable) fast shared file system support to the cluster through 4 IBM x342 servers using IBM's General Parallel File System (GPFS). This additional disk space will also be made available to all current HPCC machines via the network using new Gigabit network connections added to those machines.

Also added this summer was a Sun V880 server (sun9.hpcc.nd.edu). This machine contains 8 900 MHz UltraSPARC III processors, an 8 MB L2 cache, 16 GB RAM and a 140 GB striped & mirrored local disk. This machine provides significantly faster compute resources for the Sun
architecture.

Network connectivity between all HPCC machines was significantly
improved with the addition of a new Cisco 6509 switch providing Gb
network interfaces for most machines. This switch is configured to use jumbo frames (where possible) and the IBM Linux nodes use High Performance SysKonnect SK-9821 network cards for high bandwidth.

Please see the HPCC facilities web page at http://www.nd.edu/~hpcc/facilities for more details. Of interest may
be the diagram of the HPCC Network, this can be found at the end of
the HPCC Facilities web page.

**********************************************
Early User Test Period
**********************************************

Although we had initially targeted July 15th for the start of "Early User Test Period" unfortunately some delays were experienced with the new hardware and especially GPFS.

At this time we'd like to announce the availability of the Sun V880
(sun9), and 7 IBM x330 dual processor nodes (cnode02-8.hpcc.nd.edu) through the batch system. Note that these are currently configured as individual 2 processor SMP machines. We'll soon be working on providing multinode Gaussian support via Linda. We are ready to add additional Linux nodes if usage demands it but would like to reserve some nodes for MPI testing, etc. Please watch the motd for updates.

Please note that the IBM Linux nodes represent another hardware
architecture for users. Due to this, one of the cluster nodes is
available for interactive use and is similar to the SGI & Sun
interactive front ends. The name of the Linux interactive machine is
linux1.hpcc.nd.edu. Please note that linux1.hpcc.nd.edu is an alias to
cnode01.hpcc.nd.edu which may need to be used for some versions of
ssh. When submitting batch jobs the architecture type is
-l arch=glinux All Linux nodes available are currently running
Red Hat 7.2. (2.4.9.34smp)

During "Early User Test Period" the new hardware and new infrastructure may be more susceptible to instabilities and possibly even unannounced outages for reinstallations, reconfiguration, reboots, etc. We will communicate problems/outages through the motd as they occur.

Due to instabilities encountered with the distributed GPFS file system
this will only be available on the IBM Linux hardware until we gain
more experience and confidence with this product.

**********************************************
Linux Software Available
**********************************************

In addition to the installation and testing of hardware and HPCC
infrastructure, an AFS directory infrastructure has been created to
support applications similar to the SGI & Sun environments.

The HPCC Linux space will utilize modules for flexible selection of
various versions of software. The software which is currently
available is listed below.

Adobe Acrobat Version 5.0.5
AFS - OpenAFS Version 1.2.6
Fluent Inc. - Fidap Version 8.6.2
Fluent Inc. - Gambit Version 2.04
Gaussian - G98A11.3 (Running as a 2 processor SMP job)
GCC & G77 Version 2.96 - Part of the Linux OS
Intel - C/C++ & Fortan Compilers Version 6.0
LAM-MPI Version 6.5.6
Maple Version 7
Matlab Version 6.0, 6.1 & 6.5
Mathmatica Version 4.1
Modules Version 3.1.6
MPICH 1.2.4
Portland Group Cluster Development Kit Version 3.3 & Version 4.0 ( 2 x 16 node licenses)
Includes:
* Floating multi-user seats for PGI's parallel Fortran, C, and
C++ compilers for Linux -- industry-leading single-processor
performance and integrated native support for all 3 popular
parallel programming models: HPF, OpenMP, and MPI.
* Graphical MPI and OpenMP Linux Cluster debugging (PGDBG) and
parallel performance profiling (PGPROF) tools.
* Pre-compiled/pre-configured MPI-CH message-passing libraries
and utilities
* Optimized BLAS and LAPACK serial math libraries
* Pre-compiled ScaLAPACK parallel math library
* Tutorial examples and programs to help you get your codes up
and running quickly using HPF, OpenMP, and MPI messaging
* OpenMP & MPI Debugger

Research Systems - IDL Version 5.5
SAS (on order)
Sun Grid Engine (SGE aka GRD) Batch software Version 5.3

**********************************************
Linux Documentation
**********************************************

At this time there is documentation for the Portland Group & Intel
compilers; this can be found under Linux off the HPCC web page. We'll be working on adding more documentation and software, as requested in the User Survey from earlier this year.

**********************************************
Feedback / Help
**********************************************

Please contact us with feedback using the e-mail address of hpcc@nd.edu this gets sent to both of us and helps should one of us be on vacation, etc.

Rich Sudlow & In-Saeng Suh

page modified 12/12/02