RutgersTHE STATE UNIVERSITY OF NEW JERSEY
Department of Physics & Astronomy
Center for Materials Theory
← Cluster overview

Jupyter

Historical Jupyter procedure. The Anaconda2 modules, rupc02/rupc07 hosts, and configuration examples below describe the previous environment. They have not been validated on the Rocky Linux 9 cluster. Ask the administrator for the supported Python module and scheduler-approved way to run Jupyter; do not run interactive jobs on the login node. For remote access, start with VPN configuration.

Current upstream reference: Running Jupyter Notebook. Cluster-specific launch instructions will be updated separately.

Jupyter Notebook Configuration and Usage on RUPC

Configuration

Introduction

A traditional way to start Jupyter Notebook on a cluster is to connect using ssh -Y and enter the 
jupyter-notebook
command in the terminal. After a minute or two, a browser window should appear. A browser running on the cluster may respond slowly, depending on network speed and system load. The approach below uses a virtual private network (VPN) to connect your computer to the cluster over a private network. You can then use your local browser to access Jupyter Notebook running on the cluster, reducing the amount of data transferred. This approach uses the private network for cluster access and does not require the firewall configuration described for public access.

Prerequisites

To use Jupyter Notebook on the cluster, install OpenVPN on your computer. To install OpenVPN, follow these steps . OpenVPN provides a secure private connection to the cluster from any location with network access.

One-Time Configuration on RUPC

To configure the Jupyter Notebook server, log in to the RUPC cluster using rupc02. Then follow these steps to set up and run Jupyter Notebook.
  1. Load the Anaconda2 module: 
    module load anaconda2
  2. Generate the Jupyter Notebook configuration file: 
    cd ~
jupyter notebook --generate-config

    This command will create a default Jupyter Notebook configuration file: /home/user/.jupyter/jupyter_notebook_config.py, where user should be replaced with your username. Replace user with your username in all examples and commands below.

  3. For security, set a password for your Jupyter Notebook server: 
    python
>>> from notebook.auth import passwd
>>> passwd()
Enter password:<your-password>
Verify password:<your-password>
'sha1:<your-sha1-hash-value>'
>>> Ctrl+Z

    Save the SHA1 hash for later use. It will look like this:

    sha1:44add4a345cc:bdr1fb6343557bff721e65e78fc511c41b13e8be.

  4. Create a self-signed certificate and a matching key: 
    mkdir ~/certs
chmod 700 ~/certs
cd ~/certs
openssl req -x509 -nodes -days 1095 -newkey rsa:2048 -keyout jupyterkey.key -out jupytercert.pem

    The last command will generate a certificate file /home/user/certs/jupytercert.pem and the matching key file /home/user/certs/jupyterkey.key.

    Example command output: 
    Generating a 2048 bit RSA private key
..................................................................................................................................................................................++
................................................................................................................................................................................................................................................................++
writing new private key to 'jupyterkey.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) [ ]:NJ
Locality Name (eg, city) [Default City]:Piscataway
Organization Name (eg, company) [Default Company Ltd]:Rutgers
Organizational Unit Name (eg, section) [ ]:Physics
Common Name (eg, your name or your server's hostname) [ ]:rupc02
  5. Open the default configuration file in a text editor, such as vi: 
    vi /home/user/.jupyter/jupyter_notebook_config.py
  6. Find the following lines, uncomment them, and modify them as shown below. The result should look like this (remember to replace user with your username.): 
    c.NotebookApp.certfile = '/home/user/certs/jupytercert.pem'
c.NotebookApp.keyfile = '/home/user/certs/jupyterkey.key'
c.NotebookApp.ip = '10.10.10.2'
c.NotebookApp.open_browser = False
c.NotebookApp.password = 'sha1:<your-sha1-hash-value-from-step-#3>'

    Save the configuration file and exit the editor.

Running Jupyter Notebook

On RUPC

On your computer

Quick Way to Run Jupyter Notebook

This method runs Jupyter Notebook without a password or an encrypted web connection. Files accessible through the notebook are therefore not protected by a web login. You must also specify the --ip address parameter on the command line each time you start the notebook.

On RUPC

Continue with the steps in the previous section, "On your computer".

Remarks

To run Jupyter Notebook with Python 3, load the corresponding Anaconda module and follow the same steps. See the example output below: 

[20:19:06]udo@rupc02:~>module load anaconda3
[20:19:16]udo@rupc02:~>jupyter-notebook 
[I 20:19:23.917 NotebookApp] Serving notebooks from local directory: /mnt/home/udo
[I 20:19:23.917 NotebookApp] 0 active kernels 
[I 20:19:23.917 NotebookApp] The Jupyter Notebook is running at:  https://10.10.10.2:8888/
[I 20:19:23.918 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 20:19:53.784 NotebookApp] 302 GET / (10.2.0.18) 0.47ms
[W 20:19:53.944 NotebookApp] /mnt/home/udo/.#1.chpl doesn't exist
[W 20:19:54.098 NotebookApp] /mnt/home/udo/.#1.chpl doesn't exist
[I 20:20:36.392 NotebookApp] 302 GET / (10.2.0.18) 0.35ms
[W 20:20:36.905 NotebookApp] /mnt/home/udo/.#1.chpl doesn't exist
[I 20:20:54.462 NotebookApp] Creating new notebook in 
[I 20:20:55.280 NotebookApp] Kernel started: f89ab83a-c69a-40c3-adba-74e0df9d0001
[I 20:21:27.852 NotebookApp] Saving file at /Untitled.ipynb

^C

[I 20:21:35.010 NotebookApp] interrupted
Serving notebooks from local directory: /mnt/home/udo
1 active kernels 
The Jupyter Notebook is running at: https://10.10.10.2:8888/
Shutdown this notebook server (y/[n])? y
[C 20:21:36.715 NotebookApp] Shutdown confirmed
[I 20:21:36.716 NotebookApp] Shutting down kernels
[I 20:21:37.118 NotebookApp] Kernel shutdown: f89ab83a-c69a-40c3-adba-74e0df9d0001
The example shows how to load anaconda3, start Jupyter Notebook, create a notebook, and stop the server using CTRL-C. Clicking the Quit button on the Jupyter Notebook home screen is the preferred way to stop the server. Jupyter Notebook is also available with the standard Python installation on rupc02 and rupc07. You can start it without loading a module by entering the command shown above: 
jupyter notebook
if you have configured Jupyter Notebook on RUPC, or 
jupyter notebook  --no-browser --ip 10.10.10.2
if you have not configured it.

Useful links

  1. Running a notebook server
  2. The Jupyter Notebook
  3. Jupyter Notebook Quickstart
  4. Set Up a Jupyter Notebook Server — CentOS/RHEL 7