Bridges User Guide
There are a variety of file transfer methods available for Bridges:
- scp or sftp
- cp - can be used between Bridges' filesystems, but only if the transfer is under 100MB
If you are required to use Two-factor authentication to access Bridges' filesystems, see Two-factor Authentication for instructions on file transfer.
Paths for Bridges file spaces
For all file transfer methods other than cp, you must always use the full path for your Bridges files. The start of the full paths for your Bridges directories are:
Home directory /home/username
Pylon2 directory /pylon2/groupname/username
Pylon5 directory /pylon5/groupname/username
id -Gn will show all of your valid groupnames. You have a pylon2 and pylon5 directory for each grant you have.
Transfers into your Bridges home directory
Your home directory quota is 10GB, so large files cannot be stored there; they should be copied into one of your pylon file spaces instead. Exceeding your home directory quota will prevent you from writing more data into your home directory and will adversely impact other operations you might want to perform.
Globus can be used for any file transfer to Bridges. It tracks the progress of the transfer and retries when there is a failure; this makes it especially useful for transfers involving large files or many files. This includes transfers between the pylon2 and pylon5 filesystems and file transfers between the Data Supercell and pylon2 or pylon5.
Once you have the proper authentication you can initiate file transfers from the Globus site. A Globus transfer requires a Globus endpoint, a file path and a file name for both the source and destination. The endpoints for Bridges are:
- psc#bridges-xsede if you are using an XSEDE User Portal account for authentication
- psc#bridges-cilogon if you are using InCommon for authentication
These endpoints are owned by firstname.lastname@example.org. If you use DUO MFA for your XSEDE authentication, you do not need to because you cannot use it with Globus. You must always specify a full path for the Bridges file systems. See Paths for Bridges file spaces for details.
The globus-url-copy command can be used if you have access to Globus client software. Both the
myproxy-logon commands are available on Bridges, and can be used for file transfers internal to the PSC.
globus-url-copy you must have a current user proxy certificate. The command
grid-proxy-info will tell you if you have current user proxy certificate and if so, what the remaining life of your certificate is.
myproxy-logon command to get a valid user proxy certificate if any one of these applies:
- you get an error from the
- you do not have a current user proxy certificate
- the remaining life of your certificate is not sufficient for your planned file transfer
When prompted for your MyProxy passphrase enter your XSEDE Portal password.
globus-url-copy for transfers to a machine you must know the Grid FTP server address. The Grid FTP server address for Bridges is
The use of
globus-url-copy always requires full paths. See Paths for Bridges file spaces for details.
To use scp for a file transfer you must specify a source and destination for your transfer. The format for either source or destination is
For transfers involving Bridges, username is your PSC username. The machine-name should be given as data.bridges.psc.edu. This is the name for a high-speed data connector at PSC. We recommend using it for all file transfers using scp involving Bridges. Using it prevents file transfers from disrupting interactive use on Bridges' login nodes.
File transfers using scp must specify full paths for Bridges file systems. See Paths for Bridges file spaces for details.
To use sftp, first connect to the remote machine:
When Bridges is the remote machine, use your PSC userid as username. The Bridges machine-name should be specified as data.bridges.psc.edu. This is the name for a high-speed data connector at PSC. We recommend using it for all file transfers using sftp involving Bridges. Using it prevents file transfers from disrupting interactive use on Bridges' login nodes.
You will be prompted for your password on the remote machine. If Bridges is the remote machine enter your PSC password.
You can then enter sftp subcommands, like
put to copy a file from the local system to the remote system, or
get to copy a file from the remote system to the local system.
To copy files into Bridges you must either
cd to the proper directory or use full pathnames in your file transfer commands. See Paths for Bridges file spaces for details.
If you are required to use two-factor authentication (TFA) to access Bridges' filesystems, you must enroll in XSEDE DUO. Once that is complete, use scp or sftp to transfer files to/from Bridges.
TFA users must use port 2222 and XSEDE Portal usernames and passwords. The machine name for these transfers is data.bridges.psc.edu.
In the examples below, myfile is the local filename, XSEDE-username is your XSEDE Portal username and /path/to/file is the full path to the file on a Bridges filesystem. Note that -P ( capital P) is necessary.
Transfer a file from a local machine to Bridges:
scp -P 2222 myfile XSEDEemail@example.com:/path/to/file
Transfer a file from Bridges to a local machine:
scp -P 2222 XSEDEfirstname.lastname@example.org:/path/to/file myfile
Use sftp interactively:
sftp -P 2222 XSEDEemail@example.com
Then use the
put command to copy a file from the local machine to Bridges, or the
get command to transfer a file from Bridges to the local machine.
Graphical SSH client
If you are using a graphic SSH client, configure it to connect to data.bridges.psc.edu on port 2222/TCP. Login using your XSEDE Portal username and password.
PSC maintains a Web page at http://speedpage.psc.edu that lists average data transfer rates between all XSEDE resources. If your data transfer rates are lower than these average rates or you believe that your file transfer performance is subpar, send email to firstname.lastname@example.org. We will examine approaches for improving your file transfer performance.