Golem

To support the file repository needs of its users, PSC has deployed a file archival system named golem.

Golem is a complex tape-and-disk system. Initially, files stored to golem reside on disk. Factors such as file size and time of last access determine when a file gets migrated to tape. When you access a migrated file, it is automatically read in from tape. The system is designed to insure file security through redundancy and to provide rapid file storage and retrieval, even when a file is on tape.

Golem passwords

Your golem password is your PSC Kerberos password. Initially, it is the password given on your PSC initial account form. To change your PSC password, you can use the kpasswd command on bigben, pople, salk or warhol or use the web-based PSC password change utility.

Transferring files

Golem is strictly a file archiver. It is not used for computing. Thus, you cannot login to golem to perform file transfers. Logged into another system, either at PSC or at your remote site, there are several methods you can use to transfer files to and from golem.

To transfer files from a PSC machine to golem you should use the far program. The far program should only be used interactively.

To transfer files between your local machine and golem you can use either the scp or kftp programs or, if your local machine is on the TeraGrid, the globus-url-copy command.

If you are going to store a file to golem that is 2 Terabytes or larger or if you are going to store more than 500 Gbytes in a day, send email to remarks@psc.edu so that special arrangements can be made to handle your file requests.

Improving your file transfer performance

If you are not going to use the far program, because you are not transferring files from a PSC system, then, before you select a file transfer method, you should perform two steps. First, there are certain TCP tuning operations that will improve your file transfer performance no matter which of the other three file transfer methods described above you choose. You will probably need the assistance of a network administrator at your site to perform these tuning operations.

Second, you should determine the receive buffer size for your machine. The receive buffer size for golem is 16 Mbytes. You should always perform large file transfers in the direction of the machine with the largest receive buffer size. You will probably need the assistance of a network administrator at your site to determine the size of your machine's receive buffers and to increase it, if possible.

Once you have performed these two steps, you must select a file transfer method. If your machine is on the TeraGrid, to get the best the file transfer performance you should use the globus-url-copy command. If you are not on the TeraGrid but do have Kerberos authorization available on your machine you should use kftp. Otherwise you should use the scp program. If you use scp you should install the hpn-ssh patches. You will probably need the assistance of a network administrator at your site to install these patches.

Choosing a file transfer method is the final step you can take to improve your file transfer performance. There are no options to any of the above three commands which will impact your file transfer performance. If you have questions about the recommendations in this section or believe that your performance after following these recommendations is still substandard compared to your prior results on golem or other machines send email to remarks@psc.edu. Sometimes file transfer performance can be improved by routing changes made by network administrators, either at your site or at PSC.