Pittsburgh Supercomputing Center 

Advancing the state-of-the-art in high-performance computing,
communications and data analytics.


Allocation requests being accepted through Oct. 15     More information  

Greenfield User Guide


File storage and transfer 


File spaces

There are several distinct file spaces available from Greenfield.

Your Greenfield home directory is created the first time you log in to Greenfield.
This is the default space for all your files. You have a home directory named /home/user-name, where user-name is your PSC username. The /home filesystem is shared across Greenfield, so you can access your files in /home from all Greenfield nodes. 
Your home directory is /home/user-name.
The environment variable $HOME points to your Greenfield home directory.  
There is a quota of 5GB on your home directory.
The /crucible filesystem is a scratch file system,  shared across all Greenfield nodes, providing a fast, temporary file area for data read or written by running jobs.  It is much faster than the /home system.
The /crucible filesystem is not a permanent storage space and it is not backed up. Files that have not been accessed for  21 days are subject to deletion.   Files in /crucible will also be deleted if we need to free up space to keep jobs running.
Be sure to move your /crucible files to your local site or to the Data Supercell as soon as possible after your job completes. 
Your /crucible home directory is  /crucible/group-name/user-name  where group-name is the 7 character name for the PSC group associated with your grant.
  Find your group name
If you have more than one grant, you will have a /crucible directory for each grant.    BE SURE to use the appropriate directory when working under multiple grants.  
The Data Supercell
The Data Supercell is a PSC-designed and built system for long-term storage and management of petabyte-scale data.  
Your home directory in the Data Supercell is /arc/users/user-name.
The Data Supercell is  mounted on Greenfield (that is, you can see your Data Supercell files by typing ls /arc/users/user-name from Greenfield)  and files can be copied easily between the Data Supercell and Greenfield.
To transfer files between Greenfield and the Data Supercell, use the PSC high-speed data conduit data.psc.xsede.org with either scp or sftp.  See the section on scp and sftp in this User Guide for details.
See the Data Supercell page for a  detailed description of the Data Supercell.

File Transfer

Globus Online, scp and sftp are all be available to transfer files to and from Greenfield.   

Our recommended method for file transfer is Globus Online. If you cannot use Globus Online, but do have access to Globus client software, we recommend globus-url-copy. Otherwise, you can use sftp or scp.

Both Globus Online and globus-url-copy use high-speed data conduits, and are much faster than other file transfer methods.


In order to use Globus Online to transfer files, you must either create a Globus Online account  or use InCommon credentials.   

If you are unable to use either Globus or InCommon credentials to authenticate to Globus, send email to This email address is being protected from spambots. You need JavaScript enabled to view it. to see if you can use other methods of authentication.

Create a Globus Online account

Go to https://www.globus.org/, click "Sign Up" in the upper right, and follow the instructions.

Once you have done that, you can use Globus Online to transfer files to Greenfield.

Use InCommon credentials

If you are affiliated with an InCommon institution you can use your username and password for that institution to authenticate to Globus.  You must also  register with PSC as an InCommon user before you can transfer files to PSC with your InCommon credentials.  

To register with PSC as an InCommon user, follow these steps.

  1. Go to https://cilogon.org/
  2. Select your institution from the 'Select an Identity Provider' list
  3. Click on the 'Log On' button

    This will take you to a login page for your institution.

  4. Enter your username and password for your institution
  5. Click on the 'Login' button

    You will be redirected back to the CILogon Service web page.

  6. Find and copy your "Certificate Subject" string

    Near the top of the CILogon Service web page you will see a field called "Certificate Subject" with a string like /DC=org/DC=cilogon/C=US/O=My Institution/CN=My Name A1234. Copy this string. You'll need it in step 9.

  7. Log off from the CILogin Service webpage
  8. Log in to https://dirs.psc.edu/cgi-bin/teragrid/userpage/list.pl with your PSC username and password

    This site lists the certificate subjects (DNs) that we have in our PSC database for your PSC account.

  9. Add your CILogon Certficate Subject (DN) to this list
    1. Click on the 'Add DN' link at the top left

      This will take you to the "Adding DN" page.

    2. Paste the certificate subject that you copied in step 6 into the DN: field

      Make sure there are no extra spaces before or after the pasted string.

    3. Click on 'Create' to add your new CILogon DN (certificate subject) to the PSC database

      You can click on the 'List DNs' link at the top left to confirm that your new DN was added.

Within an hour you should be able to use Globus Online to copy files using your username and password at your institution. Send email to This email address is being protected from spambots. You need JavaScript enabled to view it. if you have any questions about using InCommon.

Moving files with Globus Online

To start a file transfer, log in to the Globus site.  

Globus transfers files between known endpoints.  XSEDE users will use one set of endpoints and InCommon users will use another.

On the Globus site, choose 'Transfer Files' and you will be taken to a graphical interface where you will select endpoints, specify source and destination paths and identify the files to be copied. 

 The endpoints and paths to use for Globus Online are given in the following tables.   XSEDE users may need to authenticate with their XSEDE Portal username and password when they select an endpoint.
 Endpoints for XSEDE usersInCommon users
Greenfield psc#greenfield psc#dxc-cilogon
Data Supercell psc#dsc-xsede psc#dsc-cilogon

InCommon users, note the difference between the Greenfield endpoint and the Data Supercell endpoint: dxc-cilogon vs. dsc-cilogon.

Path for
Greenfield home directory /greenfield/home/user-name
Greenfield scratch directory /crucible/group-name/user-name1
Data Supercell /arc/users/user-name

1To find your group-name, log in to Greenfield and type id-Gn.   Remember that /crucible is a scratch file system and is only appropriate for temporary storage.  Files that you wish to keep should be stored in the Data Supercell.

XSEDE users can use GridFTP and the globus-url-copy command line client to transfer files to the DSC. Once on the DSC, files can moved to /crucible or /home as needed.


To use globus-url-copy, first ensure that you have a current user proxy certificate for authentication with enough time on it to complete your transfer by typing


The output will look like:

subject : /C=US/O=National Center for Supercomputing Applications/CN=Joe User
issuer : /C=US/O=National Center for Supercomputing Applications/OU=Certificate Authorities/CN=MyProxy
identity : /C=US/O=National Center for Supercomputing Applications/CN=Joe User
type : end entity credential
strength : 2048 bits
path : /tmp/x509up_u99999
timeleft : 11:58:33

If the timeleft is not sufficient, or you get an "ERROR: Couldn't find a valid proxy" message, then use myproxy-logon (or if you have your own long term user certificate, grid-proxy-init) to obtain a new user proxy certificate, e.g.:

myproxy-logon -l xsede-user-name -t 24

The output will look like:

Enter MyProxy pass phrase:
A credential has been received for user xsede-user-name in /tmp/x509up_u99999.

where xsede-user-name is your XSEDE User Portal login name, -t 24 requests a 24-hour certificate, and the MyProxy pass phrase entered is your XSEDE User Portal password.


You can then use globus-url-copy to transfer files. You will need to know:

  • The GridFTP server address for the system you want
  • The absolute path to the files


  GridFTP server address Absolute path to home directory
For Greenfield gsiftp://data.greenfield.psc.xsede.org greenfield/home/user-name
For the Data Supercell gsiftp://gridftp.psc.xsede.org arc/users/user-name

where user-name is your PSC user name.


  • Copy a file (mydata.out) from your Greenfield home directory to the DSC.  Type (note that this command must be typed all on one line):
    globus-url-copy -vb gsiftp://data.greenfield.psc.xsede.org/greenfield/home/user-name/mydata.out  gsiftp://gridftp.psc.xsede.org/arc/users/user-name/mydata.out

    The output will be similar to:

    Source: gsiftp://data.greenfield.blacklight.psc.xsede.org/user-name/
    Dest:   gsiftp://gridftp.psc.xsede.org/arc/users/user-name/
      mydata.out  ->  mydata.out

    The -vb option gives feedback on the transfer. You can omit it if you like.

scp and sftp

You can transfer files between your local systems and Greenfield using the SSH file transfer clients scp andsftp. Both graphical and command line versions of these clients are available.


Using a graphical scp or sftp app

If you have a graphical scp or sftp client application on your local system, you can use it to transfer files to Greenfield.  Connect to greenfield.psc.xsede.org and use your PSC username and password for authentication. 

Using scp from the command line

For scripted transfers, or transfers that you want to execute directly from your command-line shell, you can use the SSH scp client.  

General scp use

The general format of the scp command is:

scp source-file destination-file

where the full specification for both source-file and destination-file is


The machine-name for Greenfield is greenfield.psc.xsede.org.

Luckily most of the time you can rely on default settings to avoid typing the full specification. For example,  you can omit:

  • the machine-name of the local machine
  • the user-name if it is the same on both source and destination machine
  • the path-to-file when it is  your home directory on the remote machine  
  • the path-to-file when it is your current directory on the local machine
  • the destination filename unless you want to rename it

Transfer files from blacklight to Greenfield home directories

We recommend that you use scp to transfer files from your blacklight home directory to your Greenfield home directory. 

To copy a file from your blacklight home directory to your current Greenfield directory:

  1. log in to Greenfield
  2. move to the directory where you want to store the file
  3. type
    scp tg-login1.blacklight.psc.teragrid.org:filename  .

Transfer files between Greenfield and the Data Supercell

Use the command line scp client to copy files between Greenfield and  the Data Supercell using the PSC high-speed data conduit data.psc.xsede.org.

To copy filename to your Data Supercell home directory, type

scp filename user-name@data.psc.xsede.org:filename


Using sftp from the command line

Transfer files to the Data Supercell

Use the command line sftp client to copy files to and from the Data Supercell using the PSC high-speed data conduit data.psc.xsede.org.

  1. First authenticate to data.psc.xsede.org
    sftp user-name@data.psc.xsede.org

    You will be prompted for your PSC password. The first time you connect to data.psc.xsede.org, you will be asked to accept the server's host key.

    Your default directory on data.psc.xsede.org is your Data Supercell home directory, /arc/users/user-name

  2. Use the put command to copy files into the Data Supercell and the get command to copy files from the Data Supercell.
    put filename .

    puts filename into your Data Supercell directory.

    get filename .

    copies filename from the Data Supercell into your current directory.


Transfer files to your Greenfield home directory

You can use the command line sftp client to transfer files to and from Greenfield interactively.  

  1. First authenticate to greenfield.psc.xsede.org using a command like
      sftp user-name@greenfield.psc.xsede.org

    The first time you connect to greenfield.psc.xsede.org using sftp, you may be prompted to accept the server's host key.

    You will then be prompted to enter your PSC password.

  2. Transfer files using the put command to copy a file from your local system to Greenfield, or get to copy a file from Greenfield to your local system. Your default directory for sftp to greenfield.psc.xsede.org is your Greenfield home directory.

Other sftp commands

At the sftp> prompt, you can use other sftp  commands to manage and transfer your files to/from Greenfield. Enter a question mark for a list of available sftp commands.

  See more sftp examples

Examples (entered commands appear in bold):

  • What directory am I in on Greenfield?
    sftp> pwd
    Remote working directory: /home/user-name
  • What directory am I in on my local system?
    sftp> lpwd
    Local working directory: /Users/JoeUser/Documents
  • Change directories on my local system to /Users/JoeUser/Documents/project1:
    sftp> lcd /Users/JoeUser/Documents/project1
  • Make a new directory called "newdata" under my current directory on Greenfield :
    sftp> mkdir newdata
  • Exit from this sftp session :
    sftp> exit


Spring2015 cover

View all PSC publications

PSC Fall2014 cover PSC Spring2014cPSC2013 covers web 

Subscriptions: You can receive PSC news releases via e-mail. Send a blank e-mail to psc-wire-join@psc.edu.