Pittsburgh Supercomputing Center 

Advancing the state-of-the-art in high-performance computing,
communications and data analytics.

File spaces

There are several distinct file spaces available from Greenfield.

/home
This is the default space for all your files. You have a home directory named /home/username, where username is your PSC username. The /home filesystem is shared across Greenfield, so you can access your files in /home from all Greenfield nodes. There is currently no quota on the /home file system.  
Your home directory is /home/username.
Crucible
The crucible filesystem is a scratch file system,  shared across all Greenfield nodes, providing a fast, temporary file area for your running jobs.  It is much faster than the /home system.
Crucible is not a permanent storage space and it is not backed up. Files will only remain on crucible for up to 21 days and then they will be deleted.   Crucible files will also be deleted if we need to free up space to keep jobs running.
Be sure to store copies of your crucible files to your local site or to the Data Supercell as soon as possible after you create them. 
Your crucible home directory is  /crucible/group-name/user-name  where group-name is the 7 character name for the PSC group associated with your grant.
Find your group name
The environment variable $SCRATCH points to your crucible home directory.
If you have more than one grant, you will have a crucible directory for each grant.  $SCRATCH will point to the directory associated with your default grant.  BE SURE to use the appropriate directory when working under multiple grants.  
The Data Supercell
The Data Supercell is a PSC-designed and built system for long-term storage and management of petabyte-scale data.  It is mounted on all Greenfield nodes.  Files can be copied easily between the Data Supercell and Greenfield as needed.
Your home directory in the Data Supercell is /arc/users/username.
See the Data Supercell page for a  detailed description.

File Transfer

Globus Online, scp and sftp will all be available to transfer files from Blacklight to Greenfield.   More information will be available here for globus-url-copy, scp and sftp as it becomes available.

Our recommended method for file transfer is Globus Online. If you cannot use Globus Online, but do have access to Globus client software, we recommend globus-url-copy. Otherwise, you can use sftp or scp.

In order to use Globus Online to transfer files, you must either create a Globus Online account  or use InCommon credentials.   

Create a Globus Online account

Go to https://www.globus.org/, click "Sign Up" in the upper right, and follow the instructions.

Once you have done that, you can use Globus Online to transfer files to Greenfield.

Use InCommon credentials

If you are affiliated with an InCommon institution you can use your username and password for that institution to authenticate to Globus.  You must also  register with PSC as an InCommon user before you can transfer files to PSC with your InCommon credentials.  

To register with PSC as an InCommon user, follow these steps.

  1. Go to https://cilogon.org/
  2. Select your institution from the 'Select an Identity Provider' list
  3. Click on the 'Log On' button

    This will take you to a login page for your institution.

  4. Enter your username and password for your institution
  5. Click on the 'Login' button

    You will be redirected back to the CILogon Service web page.

  6. Find and copy your "Certificate Subject" string

    Near the top of the CILogon Service web page you will see a field called "Certificate Subject" with a string like /DC=org/DC=cilogon/C=US/O=My Institution/CN=My Name A1234. Copy this string. You'll need it in step 9.

  7. Log off from the CILogin Service webpage
  8. Log in to https://dirs.psc.edu/cgi-bin/teragrid/userpage/list.pl with your PSC username and password

    This site lists the certificate subjects (DNs) that we have in our PSC database for your PSC account.

  9. Add your CILogon Certficate Subject (DN) to this list
    1. Click on the 'Add DN' link at the top left

      This will take you to the "Adding DN" page.

    2. Paste the certificate subject that you copied in step 6 into the DN: field

      Make sure there are no extra spaces before or after the pasted string.

    3. Click on 'Create' to add your new CILogon DN (certificate subject) to the PSC database

      You can click on the 'List DNs' link at the top left to confirm that your new DN was added.

Within an hour you should be able to use Globus Online to copy files using your username and password at your institution. Send email to This email address is being protected from spambots. You need JavaScript enabled to view it. if you have any questions about using InCommon.

Moving files with Globus Online

You can transfer files to the Data Supercell through Globus Online.  To start a file transfer, log in to the Globus site.  

Globus transfers files between known endpoints.  XSEDE users will use one set of endpoints and InCommon users will use another.

On the Globus site, choose 'Transfer Files' and you will be taken to a graphical interface where you will select endpoints and identify the files to be copied.

For XSEDE users, the endpoint for the Data Supercell is psc#dxc-xsede.    You may need to authenticate with your XSEDE Portal username and password when you select this endpoint.

For InCommon users, the endpoint for the Data Supercell is psc#dxc-cilogon.

In both cases, be sure to specify a destination path for where your files should be stored on Greenfield.  

If you are unable to use either Globus or InCommon credentials to authenticate to Globus, send email to This email address is being protected from spambots. You need JavaScript enabled to view it. to see if you can use other methods of authentication.

 Appropriate Globus endpoints 

Transfer files from an external site to the Data Supercell

To transfer files from an external (non-PSC) Globus endpoint to the crucible filesystem, use the external endpoint and use either psc#dsc-xsede (if you are an XSEDE user) or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell.  Be sure to specify the complete destination path, starting with /arc/users/username. 

Transfer files from the brashear filesystem on Blacklight to the Data Supercell

To transfer files from the Blacklight brashear filesystem to the crucible filesystem, use endpoint xsede#blacklight for brashear and psc#dxc-xsede (if you are an XSEDE user)  or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell.  Be sure to specify the complete destination path, starting with /arc/users/username.

Transfer files between the Data Supercell and the crucible filesystem

To transfer files between the Data Supercell and the crucible filesystem, use the endpoint xsede#blacklight (if you are an XSEDE user)  or psc#dxc-cilogon (if you are an InCommon user) for the Data Supercell and the endpoint psc#dxc-xsede for crucible.   Be sure to specify complete paths for both endpoints.  On the Data Supercell, the complete path will start with /arc/users/username.   On crucible, the complete path will start with /crucible/group-name/username.

Sftp and scp

You can transfer files between your local systems and Greenfield using the SSH file transfer clients sftp andscp. Both graphical and command line versions of these clients are available.

You should only use sftp or scp to transfer a small number of small files.  For larger files or to transfer many files,  use Globus Online.

Using a graphical sftp or scp app

If you have a graphical sftp or scp client application on your local system, you can use it to transfer files to Greenfield.  Connect to greenfield.psc.xsede.org and use your PSC username and password for authentication. 

Using sftp from the command line

You can use the command line sftp client to transfer files to and from Greenfield interactively.

  1. First authenticate to greenfield.psc.xsede.org using a command like
     $ sftp username@greenfield.psc.xsede.org

    The first time you connect to greenfield.psc.xsede.org using sftp, you may be prompted to accept the server's host key. Enter yes to accept the host key:


    The authenticity of host 'greenfield.psc.xsede.org (128.182.70.103)' can't be established.
    RSA key fingerprint is d5:77:f2:d9:07:f6:32:b6:c3:eb:0d:d1:29:ed:9b:80.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'greenfield.psc.xsede.org' (RSA) to the list of known hosts. 

    You will then be prompted to enter your PSC password.

  2. Transfer files using the put command to copy a file from your local system to Greenfield, or get to copy a file from Greenfield to your local system. Your default directory for sftp to greenfield.psc.xsede.org is your Greenfield home directory.

Other sftp commands

At the sftp> prompt, you can use other sftp  commands to manage and transfer your files to/from Greenfield. Enter a question mark for a list of available sftp commands.

  See more sftp examples

Examples (entered commands appear in bold):

  • What directory am I in on Greenfield?
    sftp> pwd
    Remote working directory: /home/username
  • What directory am I in on my local system?
    sftp> lpwd
    Local working directory: /Users/JoeUser/Documents
  • Change directories on my local system to /Users/JoeUser/Documents/project1:
    sftp> lcd /Users/JoeUser/Documents/project1
  • Make a new directory called "newdata" under my current directory on Greenfield :
    sftp> mkdir newdata
  • Exit from this sftp session :
    sftp> exit

 

Using scp from the command line

For scripted transfers, or transfers that you want to execute directly from your command-line shell, you can use the SSH scp client.

Examples (where entered commands appear in bold):

    • Copy my local file/Users/JoeUser/Documents/project1/file1.dat to my home directory on Greenfield:
      $ scp /Users/JoeUser/Documents/project1/file1.dat  username@greenfield.psc.xsede.org:.username@greenfield.psc.xsede.org's password: 
      file1.dat          100% 1016KB   1.0MB/s   00:00  
      
      The first time that you use scp to transfer files to Greenfield, you may receive a warning similar to:
      The authenticity of host '(128.182.nn.nnn)' can't be established.
      RSA key fingerprint is 05:9d:1b:98:f9:92:71:60:e7:66:bd:35:d8:89:58:d2.
      Are you sure you want to continue connecting (yes/no)? yes
      Warning: Permanently added 'greenfield.psc.xsede.org' (RSA) to the list of known hosts.
      

      You will then be prompted for your PSC password.

    • Copy all the files in my newdata directory on Greenfield to directory /Users/JoeUser/Documents/project1/newdata on my local system.  If directory /Users/JoeUser/Documents/project1/newdata does not exist, it will be created.
      $ scp -r username@greenfield.psc.xsede.org:newdata /Users/JoeUser/Documents/project1   username@greenfield.psc.xsede.org's password: 
      file2.dat          100% 1016KB   1.0MB/s   00:00
      file3.dat          100% 1016KB   1.0MB/s   00:01
      file1.dat          100% 1016KB   1.0MB/s   00:00
      

 

 

 


Publications

Spring2015 cover

View all PSC publications

PSC Fall2014 cover PSC Spring2014cPSC2013 covers web 

Subscriptions: You can receive PSC news releases via e-mail. Send a blank e-mail to psc-wire-join@psc.edu.