Getting Started with High Performance Computing
If you are new to high performance computing, there are some concepts that you should be familiar with before you start to use Bridges-2, including
- the difference between login nodes and compute nodes
- connecting to a remote resource
- interactive sessions and batch jobs
- Unix shells
When you are familiar with all of these, you are ready to connect to Bridges-2 and begin your work. See the Connecting to Bridges-2 section of the User Guide to get started.
Login vs. compute nodes
Bridges-2 resources are divided between login nodes and compute nodes. There are over 800 compute nodes of different types, but only a small number of login nodes.
Login nodes are used for administrative tasks like copying, editing and transferring files. When you connect to Bridges-2 via an ssh client (more below on ssh), the XSEDE single sign on portal, or OnDemand, you are connecting to one of the login nodes.
You cannot do your research computing on Bridges-2’s login nodes. From the login node you can either connect to the compute nodes in an interactive session or submit a batch job to the compute nodes (more on batch jobs and interactive sessions below).
Compute nodes are where your real work will be done. From a login node, you can connect to one or more compute nodes in an interactive session, or submit a batch script to be run on one or more compute nodes.
You cannot log in to the compute nodes directly. Doing so would have a detrimental effect on their performance and on any jobs running on them.
Connecting to Bridges-2
You will always connect directly to a login node, and access the compute nodes from there. For Bridges-2 specifically, there are three ways to connect: OnDemand, SSH, and XSEDE single sign-on. For specific information on connecting to Bridges-2, see the Connecting to Bridges-2 section of the Bridges-2 User Guide.
OnDemand is a web browser interface that allows you to do many tasks normally done through a command line interface through a graphical interface in your browser instead. See the OnDemand section of the Bridges-2 User Guide for more information.
SSH is a program that enables secure logins over an unsecure network. It encrypts the data passing both ways so that if it is intercepted it cannot be read.
SSH is client-server software, which means that both your local computer (your laptop or desktop, for example) and the remote computer (in this case, Bridges-2) must have it installed. SSH server software is installed on all the PSC machines. You must install SSH client software on your local machine.
Free ssh clients for Macs, Windows machines and many versions of Unix are available. Popular ssh clients (GUI) include PuTTY for Windows and Cyberduck for Macs. A command line version of ssh is installed on Macs by default; if you prefer that, you can use it in the Terminal application. You can also check with your university to see if there is an ssh client that they recommend.
XSEDE single sign-on
If you are an XSEDE user, you can use the XSEDE single sign on portal instead of ssh to reach Bridges-2’s login nodes. See the XSEDE single sign on portal for information.
Interactivity vs. batch processing
Interactivity refers to a way of working that allows you to see the results of your input immediately. On Bridges-2, you can work interactively through OnDemand or start an interactive session from a command line interface. In batch processing, your job will wait in a queue until it can run, and the results will not be available until it does.
In OnDemand, applications like RStudio, Jupyter notebooks, and Matlab run in a graphical interface very similar to how they run on your laptop.
Interactive sessions can be started by logging in to a login node directly through ssh or XSEDE single sign-on. From a Bridges-2 login node, you will ask for a number of compute nodes to be allocated to you. When they are, you will be connected to the compute nodes and you will be able to type commands at the prompt and get an immediate response as soon as the command completes. When you are finished, you will type CTRL-D to end your interactive session. You will be returned to the login node.
In batch processing, you must create a file, called a batch script, that contains all the commands you want to run. The batch script is submitted to the scheduler on Bridges-2 (called SLURM, more about that in the Running Jobs section of this User Guide) and goes into a queue. At this point, you can log out of Bridges-2. Your job does not need further intervention from you. It will run as soon as there is room for it. When it does, a file containing the output from the job is written in your home directory. Other files, such as data results from the job, can be created also. You can check the output files whenever you like. You do not have to remain logged in to Bridges-2 until the job completes.
For more information on working in an interactive session or submitting a batch job on Bridges-2, see the Running Jobs section of the Bridges-2 User Guide.
A Unix shell is a command-line interpreter that provides a traditional user interface for Unix and Unix-like systems. Because Bridges-2 uses a Unix-like operating system, you will be using a shell whenever you are logged in to Bridges-2 and in batch scripts. The two major shell types are the Bourne shell and the C shell. Each type has its own commands and syntax.
The default shell on Bridges-2 is bash, a Bourne-type shell. Other shells, including some C-type shells, are available for you to use if you prefer.
For the most part, you do not have to worry about which shell you are using, but you will see this term used throughout the User Guide. Where the commands to use differ depending on your shell, both bash and csh command versions will be given.