Lemieux

Lemieux

Lemieux was decommissioned on December 22, 2006.

  • The queues were turned off at midnight EST, Tuesday, December 19.
  • Logins were disabled at 5 pm EST on Friday, December 22.

See the migration document for guidance on migrating from lemieux to bigben, and read the bigben document for additional information on computing on bigben.

Contact PSC User Services with any questions.

Additional Information


Storing Files

File repositories

File repositories are file storage spaces which are not directly connected to a frontend or compute node. You cannot, for example, open a file that resides in a file repository from inside a program. You must use explicit file copy commands to move files to and from the repository. You currently have one file repository available to you on lemieux: golem.psc.edu.

Golem is a combination tape-and-disk archival system.

Transfer files to and from golem interactively before and after your batch job runs, so your job does not tie up compute nodes while performing file transfers.

If you need to store a file to golem that is 2 Tbytes or larger please first contact User Services so that special arrangements can be made to store your file.

Files can be stored and retrieved from golem using one of these methods: tcscp, far, kftp, gridftp, scp or sftp.

tcscp

The tcscp program can be used to transfer files between golem and your lemieux home directory or $SCRATCH. To transfer files between golem and $LOCAL on your compute nodes you must use either $SCRATCH or $HOME as intermediate locations.

far

The far program can be used to transfer files between golem and your lemieux home directory or $SCRATCH. To transfer files between golem and $LOCAL on your compute nodes you must use either $SCRATCH or $HOME as intermediate locations.

We recommend using tcscp rather than far for transferring files between $SCRATCH or $HOME and golem. If you prefer the far interface you can indicate that you want far to actually use tcscp for file transfers by placing the line 

set transport = "tcscp"

in the file .far.conf in your home directory.

kftp, gridftp, scp, sftp

You can use kftp, gridftp, scp or sftp to transfer files between golem and your remote machine. We recommend against using sftp. See the golem Web page for more information.

Transferring Files

You can use kftp or scp to copy files between lemieux and a remote machine. The far program transfers files from lemieux to golem, the archival system.

Both kftp and scp are secure file transfer methods. But while kftp encrypts the authentication information only, scp encrypts both the authentication information and the data being transferred. For this reason, kftp is generally faster than scp.

Kftp

Lemieux is running both a Kerberos 5 (K5) client and server. Thus, if your local machine has K5 client/server software installed, you can use kftp to transfer files to and from lemieux whether you are logged into lemieux or your local machine. The examples below assume that you are logged into your local machine.

Before you can use kftp to transfer files, you must authenticate yourself to lemieux. To do this use the kinit command. 

kinit username@PSC.EDU

For 'username' substitute your PSC userid. PSC.EDU is PSC's Kerberos realm name.

After you enter this command you are prompted for your PSC Kerberos password, which is the password you use to login to lemieux.

Once you are authenticated you can use the kftp command to actually perform your file transfers. 

kftp lemieux.psc.edu

The kftp command functions like the ftp command.

To transfer files into or from $SCRATCH you will need to specify the full pathname without using any variables.

You should verify that the Kerberos commands operate on your local system as described here. Some installations of Kerberized ftp differ in their implementation.

Man pages for kinit and kftp are available on lemieux.

A Unix kftp client is available at http://www.pdc.kth.se/heimdal. A Windows kftp client is available at http://web.mit.edu/network/kerberos-form.html.

Scp, discussed below, and sftp can also be used for lemieux file transfers, but kftp will generally be much faster. We recommend against using sftp.

Scp

The secure copy program, scp, can be used to transfer files between your remote machine and your lemieux home directory or your $SCRATCH directory. You cannot use scp to copy files between $LOCAL and your remote machine. To get remote files to $LOCAL you must go through your lemieux home directory or $SCRATCH.

The format for the scp command is 

scp source-filename target-filename
where the filename on the remote system, whether it is the target or the source, must be specified as 
username@system:filename

For example, to copy a file to your $SCRATCH directory on lemieux when you are logged in to your remote system use a command such as 

scp file username@lemieux.psc.edu:/usr/scratch/n/username/file
If you are logged in to lemieux and you want to copy over a file from your home system to $SCRATCH, use a command such as 
scp username@remote-system:file  /usr/scratch/n/username/file

The first time you use scp to or from lemieux, you will receive a message similar to 

Host key not found from list of known hosts.  Are you sure 
you want to continue connecting?

Answer 'yes' to make the connection. You should not receive this message on subsequent connections.

You will be prompted next for your password on the remote system.

You may be able to improve the scp transfer rate by choosing the blowfish encryption method rather than using the default. To do this, type:

scp -c blowfish file  username@remote-system:file

For more information on the scp command, see the scp man page.

Scp is part of the ssh distribution. PSC provides a list of sites that distribute ssh.

We strongly recommend that you use kftp rather than scp if kftp is available.

Far

You can use the far program to move files between your lemieux home directory or your $SCRATCH directory and golem, PSC's file archiver.

Tar

Whether you are transferring files between lemieux and golem or between lemieux and your remote system, if you have many files--1000 or more--it is much more efficient to tar them up into one tar file and then transfer this single tar file, especially if they are small files, 64 Kbytes or smaller

Tru64 tar--located at /bin/tar--can only create tar files up to 8 Gbytes. Gnu tar--located at /usr/psc/gnu/bin/tar--can create files larger than 8 Gbytes. However, a file created by Gnu tar that is larger than 8 Gbytes cannot be read by Tru64 tar.

You should not make a tar file larger than 50 Gbytes without first contacting User Services. You should move your tar files to golem or to your remote system as soon as you can after you create them.

Acknowledgement in Publications

PSC requests that a copy of any publication (preprint or reprint) resulting from research done on PSC systems be sent to the PSC Allocation Coordinator

Publications resulting from work done on lemieux should include a credit similar to

The computations were performed on the National Science Foundation Terascale Computing System at the Pittsburgh Supercomputing Center.