#7492 document how to deliver jars to executors

Author: scottdraves

doc/scala/SparkUI.ipynb

@@ -41,7 +41,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Optionally, the contents of the cell can produce a Spark session to fill out default values for the GUI.  Only one spark magic can be active at a time."
+    "Optionally, the contents of the cell can produce a Spark session to fill out default values for the GUI.  Only one spark magic can be connected at a time."
    ]
   },
   {
@@ -73,6 +73,33 @@
     "SparkSession.builder().master(\"local[100]\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you have added JARs to the classpath of the Spark driver with the `%classpath` magic, they can be copied to the executors as follows.  We are looking into making this automatic, and also supporting `spark.jars.packages`, see [#7498](https://github.com/twosigma/beakerx/issues/7498)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%spark\n",
+    "val jars = ClasspathManager.getJars().toArray.mkString(\",\")\n",
+    "SparkSession.builder().config(\"spark.jars\", jars)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After starting a session with a Spark cluster using one of the above configurations,\n",
+    "then code like the following runs in parallel without any additional annotation.\n",
+    "A three-way progress widget automatically appears, showing how many tasks are waiting, running, and completed."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -94,7 +121,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "By default the first 1000 rows are materialized to preview a dataset."
+    "By default the Dataset preview shows just the columns and their types.  You can click a button to materialize ten rows."
    ]
   },
   {
@@ -122,7 +149,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ds.display(1)"
+    "ds.display(1000)"
    ]
   }
  ],
@@ -139,6 +166,19 @@
    "name": "Scala",
    "nbconverter_exporter": "",
    "version": "2.11.12"
+  },
+  "toc": {
+   "base_numbering": 1,
+   "nav_menu": {},
+   "number_sections": false,
+   "sideBar": false,
+   "skip_h1_title": false,
+   "title_cell": "Table of Contents",
+   "title_sidebar": "Contents",
+   "toc_cell": false,
+   "toc_position": {},
+   "toc_section_display": false,
+   "toc_window_display": false
   }
  },
  "nbformat": 4,