Greenfield User Guide
File storage and transfer
There are several distinct file spaces available from Greenfield.
- Your Greenfield home directory is created the first time you log in to Greenfield.
- This is the default space for all your files. You have a home directory named /home/user-name, where user-name is your PSC username. The /home filesystem is shared across Greenfield, so you can access your files in /home from all Greenfield nodes.
- Your home directory is /home/user-name.
- The environment variable $HOME points to your Greenfield home directory.
- There is a quota of 10GB on your home directory.
- The /crucible filesystem is a scratch file system, shared across all Greenfield nodes, providing a fast, temporary file area for data read or written by running jobs. It is much faster than the /home system.
- The /crucible filesystem is not a permanent storage space and it is not backed up. Files that have not been accessed for 21 days are subject to deletion. Files in /crucible will also be deleted if we need to free up space to keep jobs running.
- Be sure to move your /crucible files to your local site or to the Data Supercell as soon as possible after your job completes.
- Your /crucible home directory is /crucible/group-name/user-name where group-name is the 7 character name for the PSC group associated with your grant.
- Find your group name
- If you have more than one grant, you will have a /crucible directory for each grant. BE SURE to use the appropriate directory when working under multiple grants.
- The Data Supercell
- The Data Supercell is a PSC-designed and built system for long-term storage and management of petabyte-scale data.
- Your home directory in the Data Supercell is /arc/users/user-name.
- The Data Supercell is mounted on Greenfield (that is, you can see your Data Supercell files by typing
ls /arc/users/user-namefrom Greenfield) and files can be copied easily between the Data Supercell and Greenfield.
- To transfer files between Greenfield and the Data Supercell, use the PSC high-speed data conduit data.psc.xsede.org with either scp or sftp. See the section on scp and sftp in this User Guide for details.
- See the Data Supercell User Guide for a detailed description of the Data Supercell.
Globus Online, scp and sftp are all be available to transfer files to and from Greenfield.
Our recommended method for file transfer is Globus Online. If you cannot use Globus Online, but do have access to Globus client software, we recommend globus-url-copy. Otherwise, you can use sftp or scp.
Both Globus Online and globus-url-copy use high-speed data conduits, and are much faster than other file transfer methods.