jupyter

Project jupyter is an IPython spin-off that provides a web frontend to do the coding inside your browser. It is not specific to Python but also supports kernels in other programming languages. The most notable feature is that code, markdown, equations and inline images can all be combined in the same document. The native file extension of Jupyter notebooks is .ipynb and all data is represented as JSON. Similar to IPython, Jupyter uses cells for code evaluation. This interactive approach simplifies coding, as it’s easy to incrementally add computational steps and immediately see the results. The Jupyter notebook format is especially popular in data science where it enables exploratory analysis. On the downside users need to pay attention to the execution order of the cells in order not to lose reproducibility.

Installation

JupyterLab desktop app

JupyterLab can be installed as cross-platform, standalone desktop application. It is self-contained and bundles its own Python environment with selected packages. Please refer to their documentation on how to change this environment or install additional packages.

Package installation

JupyterLab can also be installed as a normal PyPi package with pip install jupyterlab. If you prefer the classical Jupyter Notebook interface, install it with pip install notebook.

Online playground

You can try Jupyter out in your browser, without installing anything.

JupyterLite

The JupyterLite project allows to run Jupyter directly in a web browser, without even having to install Python. Of course this is only meant as playground and not for computationally intensive code. With the JupyterLite Shinx extension you can easily embed interactive code environments directly inside your documentation.

Usage

Use the jupyterlab or jupyter notebook commands to launch the server in your terminal.

It should automatically open a web browser with the Jupyter dashboard (unless you added the --no-browser option). Next use the New button on the top right to create a notebook with the Python kernel.

Refer to the documentation on how to use Jupyter on the ETH HPC clusters Euler and Leonhard.

Danger

Do not configure your Jupyter server to be publicly reachable from the internet. Without SSL encryption the password or security token will be transmitted cleartext over the network. Any eavesdropper can connect to your Jupyter and have unrestricted access to your user account and computer. The Jupyter server should only listen on localhost. Use ssh to open a browser on the remote host or connect with an ssh tunnel. We recommend to use normal Python scripts for long-running computations.

Keyboard shortcuts

Productivity can be boosted by learning the modal keyboard shortcuts to navigate and edit cells.

Edit mode:

Shift-Enter     # run cell
Tab-Tab         # code auto-completion
Esc / Ctrl-m    # switch to control mode

Control mode:

h               # view help with all keyboard shortcuts
p               # open command palette
m               # convert cell to markdown
y               # convert cell to code
dd              # delete cell
a / b           # insert new cell above/below
j / k           # move cell down/up
c / v           # copy/paste cell
n / p           # select next/previous cell
Enter / Click   # switch to edit mode

The built-in help contains many more keyboard shortcuts, which are different depending on the operating system, and therefore not listed above.

Notebooks vs Scripts

As amazing as Jupyter can be for exploratory data analysis, its JSON file format does not play well with version control and the results are only reproducible if all cells were run in the correct order. Scientists should consider using Jupyter for exploration, but switching over to Python scripts for the final computations. The scripts can trivially be run on remote machines and their changes tracked in a version control system. The VSCode editor offers a nice compromise between the interactive programming features of Jupyter and classical scripts. But you can also rely on command-line tools to convert between different formats.

Jupyter includes the nbconvert tool to convert between .ipynb and various formats (Html, LaTeX, PDF, Markdown, Python script, etc).

$ jupyter nbconvert --to python my_notebook.ipynb

A powerful third-party tool is jupytext, as used by VSCode under the hood. Notebooks can even be paired and synced with their corresponding script text files.

$ jupytext --to py:percent my_notebook.ipynb

Another approach is to use nbdev and install its git hooks for a more git-friendly Jupyter experience.