Greenfield file storage and transfer
There are several distinct file spaces available from Greenfield.
- This is the default space for all your files. You have a home directory named /home/username, where username is your PSC username. The /home filesystem is shared across Greenfield, so you can access your files in /home from all Greenfield nodes. There is currently no quota on the /home file system.
- Your home directory is /home/username.
- The crucible filesystem is a scratch file system, shared across all Greenfield nodes, providing a fast, temporary file area for your running jobs. It is much faster than the /home system.
- Crucible is not a permanent storage space and it is not backed up. Files will only remain on crucible for up to 21 days and then they will be deleted. Crucible files will also be deleted if we need to free up space to keep jobs running.
- Be sure to store copies of your crucible files to your local site or to the Data Supercell as soon as possible after you create them.
- Your crucible home directory is /crucible/group-name/user-name where group-name is the 7 character name for the PSC group associated with your grant.
- Find your group name
- The environment variable $SCRATCH points to your crucible home directory.
- If you have more than one grant, you will have a crucible directory for each grant. $SCRATCH will point to the directory associated with your default grant. BE SURE to use the appropriate directory when working under multiple grants.
- The Data Supercell
- The Data Supercell is a PSC-designed and built system for long-term storage and management of petabyte-scale data. It is mounted on all Greenfield nodes. Files can be copied easily between the Data Supercell and Greenfield as needed.
- Your home directory in the Data Supercell is /arc/users/username.
- See the Data Supercell page for a detailed description.
Globus Online, scp and sftp will all be available to transfer files from Blacklight to Greenfield. More information will be available here for globus-url-copy, scp and sftp as it becomes available.
Our recommended method for file transfer is Globus Online. If you cannot use Globus Online, but do have access to Globus client software, we recommend globus-url-copy. Otherwise, you can use sftp or scp.
In order to use Globus Online to transfer files, you must either create a Globus Online account or use InCommon credentials.
Create a Globus Online account
Go to https://www.globus.org/, click "Sign Up" in the upper right, and follow the instructions.
Once you have done that, you can use Globus Online to transfer files to Greenfield.
Use InCommon credentials
If you are affiliated with an InCommon institution you can use your username and password for that institution to authenticate to Globus. You must also register with PSC as an InCommon user before you can transfer files to PSC with your InCommon credentials.
To register with PSC as an InCommon user, follow these steps.
- Go to https://cilogon.org/
- Select your institution from the 'Select an Identity Provider' list
- Click on the 'Log On' button
This will take you to a login page for your institution.
- Enter your username and password for your institution
- Click on the 'Login' button
You will be redirected back to the CILogon Service web page.
- Find and copy your "Certificate Subject" string
Near the top of the CILogon Service web page you will see a field called "Certificate Subject" with a string like /DC=org/DC=cilogon/C=US/O=My Institution/CN=My Name A1234. Copy this string. You'll need it in step 9.
- Log off from the CILogin Service webpage
- Log in to https://dirs.psc.edu/cgi-bin/teragrid/userpage/list.pl with your PSC username and password
This site lists the certificate subjects (DNs) that we have in our PSC database for your PSC account.
- Add your CILogon Certficate Subject (DN) to this list
- Click on the 'Add DN' link at the top left
This will take you to the "Adding DN" page.
- Paste the certificate subject that you copied in step 6 into the DN: field
Make sure there are no extra spaces before or after the pasted string.
- Click on 'Create' to add your new CILogon DN (certificate subject) to the PSC database
You can click on the 'List DNs' link at the top left to confirm that your new DN was added.
- Click on the 'Add DN' link at the top left
Moving files with Globus Online
You can transfer files to the Data Supercell through Globus Online. To start a file transfer, log in to the Globus site.
Globus transfers files between known endpoints. XSEDE users will use one set of endpoints and InCommon users will use another.
On the Globus site, choose 'Transfer Files' and you will be taken to a graphical interface where you will select endpoints and identify the files to be copied.
For XSEDE users, the endpoint for the Data Supercell is psc#dxc-xsede. You may need to authenticate with your XSEDE Portal username and password when you select this endpoint.
For InCommon users, the endpoint for the Data Supercell is psc#dxc-cilogon.
In both cases, be sure to specify a destination path for where your files should be stored on Greenfield.
Appropriate Globus endpoints
Transfer files from an external site to the Data Supercell
To transfer files from an external (non-PSC) Globus endpoint to the crucible filesystem, use the external endpoint and use either psc#dsc-xsede (if you are an XSEDE user) or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell. Be sure to specify the complete destination path, starting with /arc/users/username.
Transfer files from the brashear filesystem on Blacklight to the Data Supercell
To transfer files from the Blacklight brashear filesystem to the crucible filesystem, use endpoint xsede#blacklight for brashear and psc#dxc-xsede (if you are an XSEDE user) or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell. Be sure to specify the complete destination path, starting with /arc/users/username.
Transfer files between the Data Supercell and the crucible filesystem
To transfer files between the Data Supercell and the crucible filesystem, use the endpoint xsede#blacklight (if you are an XSEDE user) or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell and the endpoint psc#dxc-xsede for crucible. Be sure to specify complete paths for both endpoints. On the Data Supercell, the complete path will start with /arc/users/username. On crucible, the complete path will start with /crucible/group-name/username.
Sftp and scp
You can transfer files between your local systems and Greenfield using the SSH file transfer clients sftp andscp. Both graphical and command line versions of these clients are available.
You should only use sftp or scp to transfer a small number of small files. For larger files or to transfer many files, use Globus Online.
Using a graphical sftp or scp app
If you have a graphical sftp or scp client application on your local system, you can use it to transfer files to Greenfield. Connect to greenfield.psc.xsede.org and use your PSC username and password for authentication.
Using sftp from the command line
You can use the command line sftp client to transfer files to and from Greenfield interactively.
- First authenticate to greenfield.psc.xsede.org using a command like
$ sftp firstname.lastname@example.org
The first time you connect to greenfield.psc.xsede.org using sftp, you may be prompted to accept the server's host key. Enter yes to accept the host key:
The authenticity of host 'greenfield.psc.xsede.org (184.108.40.206)' can't be established. RSA key fingerprint is d5:77:f2:d9:07:f6:32:b6:c3:eb:0d:d1:29:ed:9b:80. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'greenfield.psc.xsede.org' (RSA) to the list of known hosts.
You will then be prompted to enter your PSC password.
- Transfer files using the put command to copy a file from your local system to Greenfield, or get to copy a file from Greenfield to your local system. Your default directory for sftp to greenfield.psc.xsede.org is your Greenfield home directory.
Other sftp commands
At the sftp> prompt, you can use other sftp commands to manage and transfer your files to/from Greenfield. Enter a question mark for a list of available sftp commands.
Examples (entered commands appear in bold):
- What directory am I in on Greenfield?
sftp> pwd Remote working directory: /home/username
- What directory am I in on my local system?
sftp> lpwd Local working directory: /Users/JoeUser/Documents
- Change directories on my local system to /Users/JoeUser/Documents/project1:
sftp> lcd /Users/JoeUser/Documents/project1
- Make a new directory called "newdata" under my current directory on Greenfield :
sftp> mkdir newdata
- Exit from this sftp session :
Using scp from the command line
For scripted transfers, or transfers that you want to execute directly from your command-line shell, you can use the SSH scp client.
Examples (where entered commands appear in bold):
- Copy my local file/Users/JoeUser/Documents/project1/file1.dat to my home directory on Greenfield:
$ scp /Users/JoeUser/Documents/project1/file1.dat email@example.com:.firstname.lastname@example.org's password: file1.dat 100% 1016KB 1.0MB/s 00:00The first time that you use scp to transfer files to Greenfield, you may receive a warning similar to:
The authenticity of host '(128.182.nn.nnn)' can't be established. RSA key fingerprint is 05:9d:1b:98:f9:92:71:60:e7:66:bd:35:d8:89:58:d2. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'greenfield.psc.xsede.org' (RSA) to the list of known hosts.
You will then be prompted for your PSC password.
- Copy all the files in my newdata directory on Greenfield to directory /Users/JoeUser/Documents/project1/newdata on my local system. If directory /Users/JoeUser/Documents/project1/newdata does not exist, it will be created.
$ scp -r email@example.com:newdata /Users/JoeUser/Documents/project1 firstname.lastname@example.org's password: file2.dat 100% 1016KB 1.0MB/s 00:00 file3.dat 100% 1016KB 1.0MB/s 00:01 file1.dat 100% 1016KB 1.0MB/s 00:00