Overview

Teaching: 20 min
Exercises: 0 min

Questions

• How to use Jupter Lab in SuperPOD?

Objectives

• Learn port forwarding technique to enable Jupter Lab

4. Jupter Lab on SuperPOD

• There is NO display config and Open OnDemand setup in SuperPOD, so it is not quite straighforward to use Jupter Lab

• However, it is still possible to use either Jump Host or Port-Forwarding techinque in SuperPOD in order to run Jupyter Lab.

  • Jump Host is preferred method using IDE like Cursor/VSCode only
  • Port-Forwarding is an alternative method that involve the use of a loca web browser like Firefox or Chrome/Safari

4.1. Using Jump Host

• Please download and use Cursor/VSCode for all OS (Windows/Macs/Linux). From terminal, ssh to superpod as usual:

$ ssh username@superpod.smu.edu

• You will be placed into a login node: either slogin-01 or slogin-02. Request for a compute node, the configuration below can be changed according to your need. The my_allocation can be found from Cold Front project

$ srun -A my_allocation -N1 -G1 -c10 --mem=64G --time=12:00:00 --pty $SHELL

• You will be given a compute node, for example bcm-dgxa100-0001. Keep this session opened and open a new Cursor/Window.
• Click File\New Window (Ctrl+Shift+N)
• Install Remote - SSH
  • Go to the Extensions panel (Ctrl+Shift+X)
  • Search for “Remote - SSH” (by Microsoft)
  • Click Install
• Connect to a Remote Host:
  • Press Ctrl+Shift+P/or click Connect via SSH to open the Command Palette
  • Type “Remote-SSH: Connect to Host…“ and select it
  • Click “Configure SSH Host …“ to open config file
  • Enter the following script to the config file:

Host SuperPOD
    HostName superpod.smu.edu
    User username

Host GPU
    HostName bcm-dgxa100-0001
    User username
    ProxyJump SuperPOD

• Note: enter your own username. The HostName needs to change to the correct GPU nodes that you requested in previous step.
• Click on Connect to GPU
• (Select Linux for Select platform of Remote Host GPU)
• Enter Password and Duo for login node and Password again for Compute node.
• You can also select to browse the files on SuperPOD and enter Password/Duo to authenticate it.
• Once authenticated, you can work with any Jupyter Notebook from the SuperPOD directory

4.2. Using Port Forwarding

• Download and use Cursor/VSCode for all OS (Windows/Macs/Linux). From terminal, ssh to superpod with specific port, for example port 8080:

$ ssh -C -D 8080 username@superpod.smu.edu

The C stands for Compression and D stands for Dynamic port-forwarding with SOCKS4/5 to port number 8000. Feel free to change the port and remember to set it up in your browser

4.2.1 Setup browser to enable proxy viewing (similar for MacOS/Linux as well)

4.2.1.1 Using Firefox as browser:

Open Firefox, my version is 104.0.2. Use combination Alt+T+S to open up the settings tab. Scroll to bottom and select Settings from Network Settings:

• Select Manual Proxy Configuration
• In the SOCKS Host, enter localhost, Port 8000
• Check SOCKS v5.
• Check Proxy DNS when using SOCKS v5.
• Check Enable DNS over HTTPS.
• Make sure everything else is unchecked, then click OK.
• Your screenshot should look like below:
  

4.2.1.2 Using Chrome/Safari as browser:

Search for proxies and set a Socks proxy with sever localhost and port 8000.

4.2.2 Test Proxy

4.2.2.1. Test Proxy using conda environment:

Go back to MobaXTerm and login into SuperPOD using regular SSH Request a compute node

$ srun -A my_allocation -N1 -G1 -c10 --mem=64G --time=12:00:00 --pty $SHELL

Load cuda, cudnn and activate any of your conda environment, for example Tensorflow_2.9 in the home directory

$ module load conda gcc/13 cuda/12 cudnn/8.9
$ conda activate ~/tensorflow_2.17   

Make sure to install jupyter

$ pip install jupyterlab

Next insert the following command:

$ jupyter notebook --ip=0.0.0.0 --no-browser
# or
$ jupyter lab --ip=0.0.0.0 --no-browser   

The following screen appears

Copy the highlighted URLs to Firefox, you will see Jupyter Notebook port forward to this:

Select TensorflowGPU29 kernel notebook and Check GPU device:

4.2.2.2 Test Proxy using docker container:

For docker container, the command line need to have 1 additional flag:

$ jupyter lab --ip=0.0.0.0 --no-browser --allow-root

You will need to replace the hostname to the name of the node that you are having:

For example in the previous command, you need to copy and paste the following line to Firefox browser:

$ http://bcm-dgxa100-0016:8888/?token=daefb1c3e2754b37b6b94b619387cb3fd9710608e0152182 

Troubleshoot for notebook requesting password

In certain case, your Jupyter Notebook requires password to be enable, you can setup the password using the command below prior to requesting jupyter lab instance:

$ jupyter notebook password   

Key Points

• Jupter Lab, Port-Forwarding