Ctrl+K

Getting Started with the P-Cluster

Getting Started with the P-Cluster#

Attention

Please make sure to find some time to go through the below material before the hackweek.

Upon completion you:

P-Cluster Overview#

The P-Cluster is very much like a high-performance computing (HPC) system where you can run interactive jobs and/or perform parallel computing. Our P-Cluster uses compute resources from the AWS cloud.

Users connect to the P-Cluster via SSH

Connect to the P-Cluster#

Generating SSH keys#

You will need to use ssh keys to log into the P-Cluster, there are no passwords. To generate ssh keys, use a tool like ssh-keygen.

ssh-keygen -q -N "" -t rsa -b 4096 -f PATH_AND_FILENAME_OF_NEW_KEY

The -q and -N "" arguments means stay quiet and do not use a passphrase (optional).

Replace PATH_AND_FILENAME_OF_NEW_KEY to something like the following:

ssh-keygen -q -N "" -t rsa -b 4096  -f ~/.ssh/id_rsa_key_PCLUSTER

Now you will have a pair of public and private keys at the specified location. The public key has the extension .pub and the private key has the same filename with no extension. So your PUBLIC KEY will be something like id_rsa_key_PCLUSTER.pub and your PRIVATE KEY will be something like id_rsa_key_PCLUSTER. These files will be together in your key directory (e.g., ~/.ssh).

Attention

Before you can connect to the P-Cluster with your private key, you must send your public SSH key to eccohackweek@gmail.com so your public key can be registered. Do not send your private key.

Connecting to the P-Cluster using SSH and your private key#

From your local Linux or Mac terminal, connect to the P-Cluster via ssh using its IP address (34.210.1.198):

ssh -i /path/to/privatekey -X USERNAME@34.210.1.198

Note your USERNAME is not the same as your email address, it will be emailed to you.

The -X option in the ssh -X command enables X11 forwarding, allowing you to run graphical applications on a remote machine and display them locally on your computer.

If you used a passphrase to create your keys then you may be prompted to enter it:

    Enter passphrase for key '/Users/USERNAME/.ssh/id_rsa':

Once you successfully log in, you will be in your home directory, /home/USERNAME/. The home directory is intended for storing analysis code, scripts, configurations, and user-specific settings, while the /efs_ecco/USERNAME/ directory is better suited for conducting model runs.

Update the .bashrc file in your P-Cluster home directory#

By default, the P-Cluster uses Bash as the shell environment. Each user has a .bashrc file in their home directory, which is automatically executed whenever a new Bash session starts. This file can be customized to set environment variables, create aliases, and configure other shell settings. Here is an example .bashrc that sets the default Bash environment on the P-Cluster. This example .bashrc can serve as a starting point for further customization (download it, place it in your home directory, and rename it to .bashrc).

# from your home directory on the p-cluster, download the example.bashrc 
wget https://raw.githubusercontent.com/ECCO-Hackweek/ecco-2024/refs/heads/main/assets/example.bashrc
# rename it to .bashrc 
mv example.bashrc .bashrc

If preferred, users can use other shells such as Csh, Ksh, or Zsh.

Contents