languages/python/jupyter/Kernels_and_Servers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kernels and Servers\n",
    "\n",
    "The most straight-forward method of accessing a Jupyter notebook server on NREL HPC is to direct your browser to our [JupyterHub instance \"Europa\"](http://europa.hpc.nrel.gov).\n",
    "However, an important caveat is that the Jupyter installation on this node is tethered to its base Anaconda environment. What this means is you do not have privileges to install modules to this python environment.\n",
    "\n",
    "There are a few methods of accessing modules you've installed in other python/conda environments within Jupyter. The primary means of doing this is to create what are known as \"**kernels**\". An alternative approach is to add an installation of Jupyter to your conda environment and run your own server, which will use the modules from that environment.\n",
    "\n",
    "## Kernels\n",
    "In order to add jupyter kernels you will need to install the `ipykernel` module to the environment from which you would like the kernel to load form:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting package metadata: done\n",
      "Solving environment: done\n",
      "\n",
      "\n",
      "==> WARNING: A newer version of conda exists. <==\n",
      "  current version: 4.6.14\n",
      "  latest version: 4.7.12\n",
      "\n",
      "Please update conda by running\n",
      "\n",
      "    $ conda update -n base -c defaults conda\n",
      "\n",
      "\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /home/mbartlet/.conda/envs/jupyter_env\n",
      "\n",
      "  added / updated specs:\n",
      "    - ipykernel\n",
      "    - python=3\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    certifi-2019.11.28         |           py38_0         148 KB  conda-forge\n",
      "    ipykernel-5.1.3            |   py38h5ca1d4c_0         160 KB  conda-forge\n",
      "    ipython-7.10.1             |   py38h5ca1d4c_0         1.1 MB  conda-forge\n",
      "    jedi-0.15.1                |           py38_0         718 KB  conda-forge\n",
      "    jupyter_client-5.3.3       |           py38_1         132 KB  conda-forge\n",
      "    jupyter_core-4.6.1         |           py38_0          70 KB  conda-forge\n",
      "    ld_impl_linux-64-2.33.1    |       h53a641e_7         653 KB  conda-forge\n",
      "    pexpect-4.7.0              |           py38_0          78 KB  conda-forge\n",
      "    pickleshare-0.7.5          |        py38_1000          12 KB  conda-forge\n",
      "    pip-19.3.1                 |           py38_0         1.9 MB  conda-forge\n",
      "    prompt_toolkit-3.0.2       |             py_0         226 KB  conda-forge\n",
      "    pygments-2.5.2             |             py_0         669 KB  conda-forge\n",
      "    python-3.8.0               |       h357f687_5        38.4 MB  conda-forge\n",
      "    pyzmq-18.1.1               |   py38h1768529_0         524 KB  conda-forge\n",
      "    setuptools-42.0.2          |           py38_0         637 KB  conda-forge\n",
      "    six-1.13.0                 |           py38_0          22 KB  conda-forge\n",
      "    tornado-6.0.3              |   py38h516909a_0         645 KB  conda-forge\n",
      "    traitlets-4.3.3            |           py38_0         134 KB  conda-forge\n",
      "    wheel-0.33.6               |           py38_0          35 KB  conda-forge\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:        46.1 MB\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-main\n",
      "  backcall           conda-forge/noarch::backcall-0.1.0-py_0\n",
      "  ca-certificates    conda-forge/linux-64::ca-certificates-2019.11.28-hecc5488_0\n",
      "  certifi            conda-forge/linux-64::certifi-2019.11.28-py38_0\n",
      "  decorator          conda-forge/noarch::decorator-4.4.1-py_0\n",
      "  ipykernel          conda-forge/linux-64::ipykernel-5.1.3-py38h5ca1d4c_0\n",
      "  ipython            conda-forge/linux-64::ipython-7.10.1-py38h5ca1d4c_0\n",
      "  ipython_genutils   conda-forge/noarch::ipython_genutils-0.2.0-py_1\n",
      "  jedi               conda-forge/linux-64::jedi-0.15.1-py38_0\n",
      "  jupyter_client     conda-forge/linux-64::jupyter_client-5.3.3-py38_1\n",
      "  jupyter_core       conda-forge/linux-64::jupyter_core-4.6.1-py38_0\n",
      "  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7\n",
      "  libffi             conda-forge/linux-64::libffi-3.2.1-he1b5a44_1006\n",
      "  libgcc-ng          conda-forge/linux-64::libgcc-ng-9.2.0-hdf63c60_0\n",
      "  libsodium          conda-forge/linux-64::libsodium-1.0.17-h516909a_0\n",
      "  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-9.2.0-hdf63c60_0\n",
      "  ncurses            conda-forge/linux-64::ncurses-6.1-hf484d3e_1002\n",
      "  openssl            conda-forge/linux-64::openssl-1.1.1d-h516909a_0\n",
      "  parso              conda-forge/noarch::parso-0.5.1-py_0\n",
      "  pexpect            conda-forge/linux-64::pexpect-4.7.0-py38_0\n",
      "  pickleshare        conda-forge/linux-64::pickleshare-0.7.5-py38_1000\n",
      "  pip                conda-forge/linux-64::pip-19.3.1-py38_0\n",
      "  prompt_toolkit     conda-forge/noarch::prompt_toolkit-3.0.2-py_0\n",
      "  ptyprocess         conda-forge/noarch::ptyprocess-0.6.0-py_1001\n",
      "  pygments           conda-forge/noarch::pygments-2.5.2-py_0\n",
      "  python             conda-forge/linux-64::python-3.8.0-h357f687_5\n",
      "  python-dateutil    conda-forge/noarch::python-dateutil-2.8.1-py_0\n",
      "  pyzmq              conda-forge/linux-64::pyzmq-18.1.1-py38h1768529_0\n",
      "  readline           conda-forge/linux-64::readline-8.0-hf8c457e_0\n",
      "  setuptools         conda-forge/linux-64::setuptools-42.0.2-py38_0\n",
      "  six                conda-forge/linux-64::six-1.13.0-py38_0\n",
      "  sqlite             conda-forge/linux-64::sqlite-3.30.1-hcee41ef_0\n",
      "  tk                 conda-forge/linux-64::tk-8.6.10-hed695b0_0\n",
      "  tornado            conda-forge/linux-64::tornado-6.0.3-py38h516909a_0\n",
      "  traitlets          conda-forge/linux-64::traitlets-4.3.3-py38_0\n",
      "  wcwidth            conda-forge/noarch::wcwidth-0.1.7-py_1\n",
      "  wheel              conda-forge/linux-64::wheel-0.33.6-py38_0\n",
      "  xz                 conda-forge/linux-64::xz-5.2.4-h14c3975_1001\n",
      "  zeromq             conda-forge/linux-64::zeromq-4.3.2-he1b5a44_2\n",
      "  zlib               conda-forge/linux-64::zlib-1.2.11-h516909a_1006\n",
      "\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "python-3.8.0         | 38.4 MB   | ##################################### | 100% \n",
      "jedi-0.15.1          | 718 KB    | ##################################### | 100% \n",
      "jupyter_client-5.3.3 | 132 KB    | ##################################### | 100% \n",
      "six-1.13.0           | 22 KB     | ##################################### | 100% \n",
      "pexpect-4.7.0        | 78 KB     | ##################################### | 100% \n",
      "pyzmq-18.1.1         | 524 KB    | ##################################### | 100% \n",
      "ld_impl_linux-64-2.3 | 653 KB    | ##################################### | 100% \n",
      "traitlets-4.3.3      | 134 KB    | ##################################### | 100% \n",
      "pygments-2.5.2       | 669 KB    | ##################################### | 100% \n",
      "tornado-6.0.3        | 645 KB    | ##################################### | 100% \n",
      "jupyter_core-4.6.1   | 70 KB     | ##################################### | 100% \n",
      "ipython-7.10.1       | 1.1 MB    | ##################################### | 100% \n",
      "wheel-0.33.6         | 35 KB     | ##################################### | 100% \n",
      "prompt_toolkit-3.0.2 | 226 KB    | ##################################### | 100% \n",
      "setuptools-42.0.2    | 637 KB    | ##################################### | 100% \n",
      "ipykernel-5.1.3      | 160 KB    | ##################################### | 100% \n",
      "pip-19.3.1           | 1.9 MB    | ##################################### | 100% \n",
      "pickleshare-0.7.5    | 12 KB     | ##################################### | 100% \n",
      "certifi-2019.11.28   | 148 KB    | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n",
      "#\n",
      "# To activate this environment, use:\n",
      "# > conda activate jupyter_env\n",
      "#\n",
      "# To deactivate an active environment, use:\n",
      "# > conda deactivate\n",
      "#\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!conda create --yes --use-local --name jupyter_env python=3 ipykernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Available kernels:\n",
      "  python3    /opt/jupyter/anaconda3/share/jupyter/kernels/python3\n"
     ]
    }
   ],
   "source": [
    "!jupyter kernelspec list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Installed kernelspec my-kernel in /home/mbartlet/.local/share/jupyter/kernels/my-kernel\n"
     ]
    }
   ],
   "source": [
    "!python -m ipykernel install --user --name \"my-kernel\" --display-name \"My Kernel\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should now see your new kernel in the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Available kernels:\n",
      "  my-kernel    /home/mbartlet/.local/share/jupyter/kernels/my-kernel\n",
      "  python3      /opt/jupyter/anaconda3/share/jupyter/kernels/python3\n"
     ]
    }
   ],
   "source": [
    "!jupyter kernelspec list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Kernel dropdown](https://i.imgur.com/80R8WS4.png)\n",
    "\n",
    "Now you should be able to click the \"Kernel\" dropdown in the menu-bar at the top and select your newly created kernel, which will use the modules you added to the associated environment:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Local Jupyter Install"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting package metadata: done\n",
      "Solving environment: done\n",
      "\n",
      "\n",
      "==> WARNING: A newer version of conda exists. <==\n",
      "  current version: 4.6.14\n",
      "  latest version: 4.7.12\n",
      "\n",
      "Please update conda by running\n",
      "\n",
      "    $ conda update -n base -c defaults conda\n",
      "\n",
      "\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /home/mbartlet/.conda/envs/jupyter_env\n",
      "\n",
      "  added / updated specs:\n",
      "    - jupyter\n",
      "    - python=3\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    dbus-1.13.6                |       he372182_0         602 KB  conda-forge\n",
      "    entrypoints-0.3            |        py38_1000          12 KB  conda-forge\n",
      "    expat-2.2.5                |    he1b5a44_1004         191 KB  conda-forge\n",
      "    fontconfig-2.13.1          |    h86ecdb6_1001         340 KB  conda-forge\n",
      "    gettext-0.19.8.1           |    hc5be6a0_1002         3.6 MB  conda-forge\n",
      "    glib-2.58.3                |py38h6f030ca_1002         3.3 MB  conda-forge\n",
      "    gst-plugins-base-1.14.5    |       h0935bb2_0         6.8 MB  conda-forge\n",
      "    gstreamer-1.14.5           |       h36ae1b5_0         4.5 MB  conda-forge\n",
      "    importlib_metadata-1.2.0   |           py38_0          40 KB  conda-forge\n",
      "    ipywidgets-7.5.1           |             py_0         101 KB  conda-forge\n",
      "    jsonschema-3.2.0           |           py38_0          90 KB  conda-forge\n",
      "    jupyter-1.0.0              |             py_2           4 KB  conda-forge\n",
      "    jupyter_console-6.0.0      |             py_0          19 KB  conda-forge\n",
      "    libclang-9.0.0             |default_hde54327_4        22.3 MB  conda-forge\n",
      "    libiconv-1.15              |    h516909a_1005         2.0 MB  conda-forge\n",
      "    libllvm9-9.0.0             |       hc9558a2_3        25.1 MB  conda-forge\n",
      "    libuuid-2.32.1             |    h14c3975_1000          26 KB  conda-forge\n",
      "    libxcb-1.13                |    h14c3975_1002         396 KB  conda-forge\n",
      "    libxkbcommon-0.9.1         |       hebb1f50_0         472 KB  conda-forge\n",
      "    libxml2-2.9.10             |       hee79883_0         1.3 MB  conda-forge\n",
      "    markupsafe-1.1.1           |   py38h516909a_0          33 KB  conda-forge\n",
      "    mistune-0.8.4              |py38h516909a_1000          53 KB  conda-forge\n",
      "    more-itertools-8.0.2       |             py_0          33 KB  conda-forge\n",
      "    nbconvert-5.6.1            |           py38_0         494 KB  conda-forge\n",
      "    notebook-6.0.1             |           py38_0         5.9 MB  conda-forge\n",
      "    nspr-4.24                  |       he1b5a44_0         1.6 MB  conda-forge\n",
      "    nss-3.47                   |       he751ad9_0         1.9 MB  conda-forge\n",
      "    pandoc-2.8.1               |                0        16.5 MB  conda-forge\n",
      "    pcre-8.43                  |       he1b5a44_0         257 KB  conda-forge\n",
      "    prompt_toolkit-2.0.10      |             py_0         221 KB  conda-forge\n",
      "    pthread-stubs-0.4          |    h14c3975_1001           5 KB  conda-forge\n",
      "    pyqt-5.12.3                |   py38hcca6a23_1         6.3 MB  conda-forge\n",
      "    pyrsistent-0.15.6          |   py38h516909a_0          90 KB  conda-forge\n",
      "    qt-5.12.5                  |       hd8c4c69_1        99.2 MB  conda-forge\n",
      "    qtconsole-4.6.0            |             py_0          90 KB  conda-forge\n",
      "    terminado-0.8.3            |           py38_0          23 KB  conda-forge\n",
      "    widgetsnbextension-3.5.1   |           py38_0         1.8 MB  conda-forge\n",
      "    xorg-libxau-1.0.9          |       h14c3975_0          13 KB  conda-forge\n",
      "    xorg-libxdmcp-1.1.3        |       h516909a_0          18 KB  conda-forge\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:       205.7 MB\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-main\n",
      "  attrs              conda-forge/noarch::attrs-19.3.0-py_0\n",
      "  backcall           conda-forge/noarch::backcall-0.1.0-py_0\n",
      "  bleach             conda-forge/noarch::bleach-3.1.0-py_0\n",
      "  ca-certificates    conda-forge/linux-64::ca-certificates-2019.11.28-hecc5488_0\n",
      "  certifi            conda-forge/linux-64::certifi-2019.11.28-py38_0\n",
      "  dbus               conda-forge/linux-64::dbus-1.13.6-he372182_0\n",
      "  decorator          conda-forge/noarch::decorator-4.4.1-py_0\n",
      "  defusedxml         conda-forge/noarch::defusedxml-0.6.0-py_0\n",
      "  entrypoints        conda-forge/linux-64::entrypoints-0.3-py38_1000\n",
      "  expat              conda-forge/linux-64::expat-2.2.5-he1b5a44_1004\n",
      "  fontconfig         conda-forge/linux-64::fontconfig-2.13.1-h86ecdb6_1001\n",
      "  freetype           conda-forge/linux-64::freetype-2.10.0-he983fc9_1\n",
      "  gettext            conda-forge/linux-64::gettext-0.19.8.1-hc5be6a0_1002\n",
      "  glib               conda-forge/linux-64::glib-2.58.3-py38h6f030ca_1002\n",
      "  gst-plugins-base   conda-forge/linux-64::gst-plugins-base-1.14.5-h0935bb2_0\n",
      "  gstreamer          conda-forge/linux-64::gstreamer-1.14.5-h36ae1b5_0\n",
      "  icu                conda-forge/linux-64::icu-64.2-he1b5a44_1\n",
      "  importlib_metadata conda-forge/linux-64::importlib_metadata-1.2.0-py38_0\n",
      "  ipykernel          conda-forge/linux-64::ipykernel-5.1.3-py38h5ca1d4c_0\n",
      "  ipython            conda-forge/linux-64::ipython-7.10.1-py38h5ca1d4c_0\n",
      "  ipython_genutils   conda-forge/noarch::ipython_genutils-0.2.0-py_1\n",
      "  ipywidgets         conda-forge/noarch::ipywidgets-7.5.1-py_0\n",
      "  jedi               conda-forge/linux-64::jedi-0.15.1-py38_0\n",
      "  jinja2             conda-forge/noarch::jinja2-2.10.3-py_0\n",
      "  jpeg               conda-forge/linux-64::jpeg-9c-h14c3975_1001\n",
      "  jsonschema         conda-forge/linux-64::jsonschema-3.2.0-py38_0\n",
      "  jupyter            conda-forge/noarch::jupyter-1.0.0-py_2\n",
      "  jupyter_client     conda-forge/linux-64::jupyter_client-5.3.3-py38_1\n",
      "  jupyter_console    conda-forge/noarch::jupyter_console-6.0.0-py_0\n",
      "  jupyter_core       conda-forge/linux-64::jupyter_core-4.6.1-py38_0\n",
      "  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7\n",
      "  libclang           conda-forge/linux-64::libclang-9.0.0-default_hde54327_4\n",
      "  libffi             conda-forge/linux-64::libffi-3.2.1-he1b5a44_1006\n",
      "  libgcc-ng          conda-forge/linux-64::libgcc-ng-9.2.0-hdf63c60_0\n",
      "  libiconv           conda-forge/linux-64::libiconv-1.15-h516909a_1005\n",
      "  libllvm9           conda-forge/linux-64::libllvm9-9.0.0-hc9558a2_3\n",
      "  libpng             conda-forge/linux-64::libpng-1.6.37-hed695b0_0\n",
      "  libsodium          conda-forge/linux-64::libsodium-1.0.17-h516909a_0\n",
      "  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-9.2.0-hdf63c60_0\n",
      "  libuuid            conda-forge/linux-64::libuuid-2.32.1-h14c3975_1000\n",
      "  libxcb             conda-forge/linux-64::libxcb-1.13-h14c3975_1002\n",
      "  libxkbcommon       conda-forge/linux-64::libxkbcommon-0.9.1-hebb1f50_0\n",
      "  libxml2            conda-forge/linux-64::libxml2-2.9.10-hee79883_0\n",
      "  markupsafe         conda-forge/linux-64::markupsafe-1.1.1-py38h516909a_0\n",
      "  mistune            conda-forge/linux-64::mistune-0.8.4-py38h516909a_1000\n",
      "  more-itertools     conda-forge/noarch::more-itertools-8.0.2-py_0\n",
      "  nbconvert          conda-forge/linux-64::nbconvert-5.6.1-py38_0\n",
      "  nbformat           conda-forge/noarch::nbformat-4.4.0-py_1\n",
      "  ncurses            conda-forge/linux-64::ncurses-6.1-hf484d3e_1002\n",
      "  notebook           conda-forge/linux-64::notebook-6.0.1-py38_0\n",
      "  nspr               conda-forge/linux-64::nspr-4.24-he1b5a44_0\n",
      "  nss                conda-forge/linux-64::nss-3.47-he751ad9_0\n",
      "  openssl            conda-forge/linux-64::openssl-1.1.1d-h516909a_0\n",
      "  pandoc             conda-forge/linux-64::pandoc-2.8.1-0\n",
      "  pandocfilters      conda-forge/noarch::pandocfilters-1.4.2-py_1\n",
      "  parso              conda-forge/noarch::parso-0.5.1-py_0\n",
      "  pcre               conda-forge/linux-64::pcre-8.43-he1b5a44_0\n",
      "  pexpect            conda-forge/linux-64::pexpect-4.7.0-py38_0\n",
      "  pickleshare        conda-forge/linux-64::pickleshare-0.7.5-py38_1000\n",
      "  pip                conda-forge/linux-64::pip-19.3.1-py38_0\n",
      "  prometheus_client  conda-forge/noarch::prometheus_client-0.7.1-py_0\n",
      "  prompt_toolkit     conda-forge/noarch::prompt_toolkit-2.0.10-py_0\n",
      "  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h14c3975_1001\n",
      "  ptyprocess         conda-forge/noarch::ptyprocess-0.6.0-py_1001\n",
      "  pygments           conda-forge/noarch::pygments-2.5.2-py_0\n",
      "  pyqt               conda-forge/linux-64::pyqt-5.12.3-py38hcca6a23_1\n",
      "  pyrsistent         conda-forge/linux-64::pyrsistent-0.15.6-py38h516909a_0\n",
      "  python             conda-forge/linux-64::python-3.8.0-h357f687_5\n",
      "  python-dateutil    conda-forge/noarch::python-dateutil-2.8.1-py_0\n",
      "  pyzmq              conda-forge/linux-64::pyzmq-18.1.1-py38h1768529_0\n",
      "  qt                 conda-forge/linux-64::qt-5.12.5-hd8c4c69_1\n",
      "  qtconsole          conda-forge/noarch::qtconsole-4.6.0-py_0\n",
      "  readline           conda-forge/linux-64::readline-8.0-hf8c457e_0\n",
      "  send2trash         conda-forge/noarch::send2trash-1.5.0-py_0\n",
      "  setuptools         conda-forge/linux-64::setuptools-42.0.2-py38_0\n",
      "  six                conda-forge/linux-64::six-1.13.0-py38_0\n",
      "  sqlite             conda-forge/linux-64::sqlite-3.30.1-hcee41ef_0\n",
      "  terminado          conda-forge/linux-64::terminado-0.8.3-py38_0\n",
      "  testpath           conda-forge/noarch::testpath-0.4.4-py_0\n",
      "  tk                 conda-forge/linux-64::tk-8.6.10-hed695b0_0\n",
      "  tornado            conda-forge/linux-64::tornado-6.0.3-py38h516909a_0\n",
      "  traitlets          conda-forge/linux-64::traitlets-4.3.3-py38_0\n",
      "  wcwidth            conda-forge/noarch::wcwidth-0.1.7-py_1\n",
      "  webencodings       conda-forge/noarch::webencodings-0.5.1-py_1\n",
      "  wheel              conda-forge/linux-64::wheel-0.33.6-py38_0\n",
      "  widgetsnbextension conda-forge/linux-64::widgetsnbextension-3.5.1-py38_0\n",
      "  xorg-libxau        conda-forge/linux-64::xorg-libxau-1.0.9-h14c3975_0\n",
      "  xorg-libxdmcp      conda-forge/linux-64::xorg-libxdmcp-1.1.3-h516909a_0\n",
      "  xz                 conda-forge/linux-64::xz-5.2.4-h14c3975_1001\n",
      "  zeromq             conda-forge/linux-64::zeromq-4.3.2-he1b5a44_2\n",
      "  zipp               conda-forge/noarch::zipp-0.6.0-py_0\n",
      "  zlib               conda-forge/linux-64::zlib-1.2.11-h516909a_1006\n",
      "\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "libiconv-1.15        | 2.0 MB    | ##################################### | 100% \n",
      "mistune-0.8.4        | 53 KB     | ##################################### | 100% \n",
      "gstreamer-1.14.5     | 4.5 MB    | ##################################### | 100% \n",
      "qt-5.12.5            | 99.2 MB   | ##################################### | 100% \n",
      "libxkbcommon-0.9.1   | 472 KB    | ##################################### | 100% \n",
      "importlib_metadata-1 | 40 KB     | ##################################### | 100% \n",
      "xorg-libxdmcp-1.1.3  | 18 KB     | ##################################### | 100% \n",
      "pyrsistent-0.15.6    | 90 KB     | ##################################### | 100% \n",
      "markupsafe-1.1.1     | 33 KB     | ##################################### | 100% \n",
      "expat-2.2.5          | 191 KB    | ##################################### | 100% \n",
      "libllvm9-9.0.0       | 25.1 MB   | ##################################### | 100% \n",
      "widgetsnbextension-3 | 1.8 MB    | ##################################### | 100% \n",
      "nspr-4.24            | 1.6 MB    | ##################################### | 100% \n",
      "libxcb-1.13          | 396 KB    | ##################################### | 100% \n",
      "libclang-9.0.0       | 22.3 MB   | ##################################### | 100% \n",
      "entrypoints-0.3      | 12 KB     | ##################################### | 100% \n",
      "pthread-stubs-0.4    | 5 KB      | ##################################### | 100% \n",
      "pyqt-5.12.3          | 6.3 MB    | ##################################### | 100% \n",
      "libuuid-2.32.1       | 26 KB     | ##################################### | 100% \n",
      "jupyter_console-6.0. | 19 KB     | ##################################### | 100% \n",
      "qtconsole-4.6.0      | 90 KB     | ##################################### | 100% \n",
      "libxml2-2.9.10       | 1.3 MB    | ##################################### | 100% \n",
      "fontconfig-2.13.1    | 340 KB    | ##################################### | 100% \n",
      "pandoc-2.8.1         | 16.5 MB   | ##################################### | 100% \n",
      "glib-2.58.3          | 3.3 MB    | ##################################### | 100% \n",
      "gst-plugins-base-1.1 | 6.8 MB    | ##################################### | 100% \n",
      "dbus-1.13.6          | 602 KB    | ##################################### | 100% \n",
      "jupyter-1.0.0        | 4 KB      | ##################################### | 100% \n",
      "terminado-0.8.3      | 23 KB     | ##################################### | 100% \n",
      "xorg-libxau-1.0.9    | 13 KB     | ##################################### | 100% \n",
      "nss-3.47             | 1.9 MB    | ##################################### | 100% \n",
      "pcre-8.43            | 257 KB    | ##################################### | 100% \n",
      "more-itertools-8.0.2 | 33 KB     | ##################################### | 100% \n",
      "jsonschema-3.2.0     | 90 KB     | ##################################### | 100% \n",
      "ipywidgets-7.5.1     | 101 KB    | ##################################### | 100% \n",
      "prompt_toolkit-2.0.1 | 221 KB    | ##################################### | 100% \n",
      "gettext-0.19.8.1     | 3.6 MB    | ##################################### | 100% \n",
      "notebook-6.0.1       | 5.9 MB    | ##################################### | 100% \n",
      "nbconvert-5.6.1      | 494 KB    | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: | b'Enabling notebook extension jupyter-js-widgets/extension...\\n      - Validating: \\x1b[32mOK\\x1b[0m\\n'\n",
      "done\n",
      "#\n",
      "# To activate this environment, use:\n",
      "# > conda activate jupyter_env\n",
      "#\n",
      "# To deactivate an active environment, use:\n",
      "# > conda deactivate\n",
      "#\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!conda create --yes --use-local --name jupyter_env python=3 jupyter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you should have a jupyter executable in your environment directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/mbartlet/.conda/envs/jupyter_env/bin/jupyter\n"
     ]
    }
   ],
   "source": [
    "%%bash -l\n",
    "module load conda\n",
    ". activate jupyter_env\n",
    "which jupyter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to start your own Jupyter server, try the following:\n",
    "```bash\n",
    "module load conda\n",
    ". activate jupyter_env\n",
    "jupyter notebook --ip=0.0.0.0 --port=8892 --no-browser\n",
    "# --no-browser prevent jupyter from trying to launch a browser on the remote machine\n",
    "# --ip=0.0.0.0 allows clients to access the server from any hostname\n",
    "```\n",
    "\n",
    "And to access the server you are running on NREL HPC, you will need to do what is called SSH tunneling:\n",
    "```bash\n",
    "my_port=8000\n",
    "remote_port=\"localhost:8892\"\n",
    "ssh -L $my_port:$remote_port el2.hpc.nrel.gov  # Be sure you ssh to the same node where you started jupyter\n",
    "```\n",
    "\n",
    "Once your SSH connection is established the ports should be tunneled and you can open this URL in your browser: http://localhost:8000\n",
    "\n",
    "You can also run a jupyter server as a SLURM job which then has access to any of the resources you requested. Here is some relevant resources by a fellow NRELian for using python to coordinate SLURM tasks:\n",
    "https://github.nrel.gov/nmerket/notebook-dask-on-eagle"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}