|
| 1 | +{ |
| 2 | + "cells": [ |
| 3 | + { |
| 4 | + "cell_type": "markdown", |
| 5 | + "metadata": {}, |
| 6 | + "source": [ |
| 7 | + "# Exercise 1 - Simple Data Parallel Example\n", |
| 8 | + "\n", |
| 9 | + "**GOAL:** The goal of this exercise is to show how to run simple tasks in parallel.\n", |
| 10 | + "\n", |
| 11 | + "The Python script used in this exercise runs too slowly, although its computation is embarrassingly parallel. Use Ray to execute the functions in parallel to speed up the script." |
| 12 | + ] |
| 13 | + }, |
| 14 | + { |
| 15 | + "cell_type": "markdown", |
| 16 | + "metadata": {}, |
| 17 | + "source": [ |
| 18 | + "### Concept for this Exercise - Remote Functions\n", |
| 19 | + "\n", |
| 20 | + "The standard way to turn a Python function into a *remote function* is to add the `@ray.remote` decorator. Here is an example.\n", |
| 21 | + "\n", |
| 22 | + "```python\n", |
| 23 | + "# A regular Python function.\n", |
| 24 | + "def regular_function():\n", |
| 25 | + " return 1\n", |
| 26 | + "\n", |
| 27 | + "# A Ray remote function.\n", |
| 28 | + "@ray.remote\n", |
| 29 | + "def remote_function():\n", |
| 30 | + " return 1\n", |
| 31 | + "```\n", |
| 32 | + "\n", |
| 33 | + "The differences are the following:\n", |
| 34 | + "\n", |
| 35 | + "1. **Invocation:** The regular version is called with `regular_function()`, whereas the remote version is called with `remote_function.remote()`.\n", |
| 36 | + "2. **Return values:** `regular_function` immediately executes and returns `1`, whereas `remote_function` immediately returns an *object ref* (a future) and then creates a task that will be executed on a worker process. The result can be obtained with `ray.get`.\n", |
| 37 | + " ```python\n", |
| 38 | + " >>> regular_function()\n", |
| 39 | + " 1\n", |
| 40 | + " \n", |
| 41 | + " >>> remote_function.remote()\n", |
| 42 | + " ObjectID(1c80d6937802cd7786ad25e50caf2f023c95e350)\n", |
| 43 | + " \n", |
| 44 | + " >>> ray.get(remote_function.remote())\n", |
| 45 | + " 1\n", |
| 46 | + " ```\n", |
| 47 | + "3. **Parallelism:** Invocations of `regular_function` happen **serially**, for example\n", |
| 48 | + " ```python\n", |
| 49 | + " # These happen serially.\n", |
| 50 | + " for _ in range(4):\n", |
| 51 | + " regular_function()\n", |
| 52 | + " ```\n", |
| 53 | + " whereas invocations of `remote_function` happen in **parallel**, for example\n", |
| 54 | + " ```python\n", |
| 55 | + " # These happen in parallel.\n", |
| 56 | + " for _ in range(4):\n", |
| 57 | + " remote_function.remote()\n", |
| 58 | + " ```" |
| 59 | + ] |
| 60 | + }, |
| 61 | + { |
| 62 | + "cell_type": "markdown", |
| 63 | + "metadata": {}, |
| 64 | + "source": [ |
| 65 | + "This example needs to run four tasks concurrently.\n", |
| 66 | + "To illustrate how these tasks run in parallel, we'll tell Ray explicitly that there are four CPUs.\n", |
| 67 | + "\n", |
| 68 | + "Usually this is not done.\n", |
| 69 | + "By default, Ray does not schedule more tasks concurrently than there are CPUs. \n", |
| 70 | + "Instead it computes the number of CPUs using `psutil.cpu_count()`:" |
| 71 | + ] |
| 72 | + }, |
| 73 | + { |
| 74 | + "cell_type": "code", |
| 75 | + "execution_count": null, |
| 76 | + "metadata": {}, |
| 77 | + "outputs": [], |
| 78 | + "source": [ |
| 79 | + "import psutil\n", |
| 80 | + "\n", |
| 81 | + "psutil.cpu_count()" |
| 82 | + ] |
| 83 | + }, |
| 84 | + { |
| 85 | + "cell_type": "markdown", |
| 86 | + "metadata": {}, |
| 87 | + "source": [ |
| 88 | + "First, let's start Ray using `ray.init` -- with the `num_cpus` set explicitly.\n", |
| 89 | + "This starts a number of processes. \n", |
| 90 | + "The other argument `ignore_reinit_error=True` simply ignores errors if this cell gets re-run multiple times." |
| 91 | + ] |
| 92 | + }, |
| 93 | + { |
| 94 | + "cell_type": "code", |
| 95 | + "execution_count": null, |
| 96 | + "metadata": {}, |
| 97 | + "outputs": [], |
| 98 | + "source": [ |
| 99 | + "import ray\n", |
| 100 | + "import time\n", |
| 101 | + "\n", |
| 102 | + "ray.init(num_cpus=4, ignore_reinit_error=True)" |
| 103 | + ] |
| 104 | + }, |
| 105 | + { |
| 106 | + "cell_type": "markdown", |
| 107 | + "metadata": {}, |
| 108 | + "source": [ |
| 109 | + "**EXERCISE:** The function below run too slow. Turn it into a remote function using the `@ray.remote` decorator." |
| 110 | + ] |
| 111 | + }, |
| 112 | + { |
| 113 | + "cell_type": "code", |
| 114 | + "execution_count": null, |
| 115 | + "metadata": {}, |
| 116 | + "outputs": [], |
| 117 | + "source": [ |
| 118 | + "# This function is a proxy for a more interesting and computationally\n", |
| 119 | + "# intensive function.\n", |
| 120 | + "def slow_function(i):\n", |
| 121 | + " time.sleep(1)\n", |
| 122 | + " return i" |
| 123 | + ] |
| 124 | + }, |
| 125 | + { |
| 126 | + "cell_type": "markdown", |
| 127 | + "metadata": {}, |
| 128 | + "source": [ |
| 129 | + "**EXERCISE:** The loop below takes too long. The four function calls could be executed in parallel.\n", |
| 130 | + "Instead of four seconds, it should only take one second for total run time.\n", |
| 131 | + "\n", |
| 132 | + "Once `slow_function` has been made a remote function, execute these four tasks in parallel by calling `slow_function.remote()`.\n", |
| 133 | + "Then obtain the results by calling `ray.get` on a list of the resulting object refs." |
| 134 | + ] |
| 135 | + }, |
| 136 | + { |
| 137 | + "cell_type": "code", |
| 138 | + "execution_count": null, |
| 139 | + "metadata": {}, |
| 140 | + "outputs": [], |
| 141 | + "source": [ |
| 142 | + "# Sleep a little to improve the accuracy of the timing measurements below.\n", |
| 143 | + "# We do this because workers may still be starting up in the background.\n", |
| 144 | + "time.sleep(2.0)\n", |
| 145 | + "start_time = time.time()\n", |
| 146 | + "\n", |
| 147 | + "results = [slow_function(i) for i in range(4)]\n", |
| 148 | + " \n", |
| 149 | + "end_time = time.time()\n", |
| 150 | + "duration = end_time - start_time\n", |
| 151 | + "\n", |
| 152 | + "print('The results are {}. This took {} seconds. Run the next cell to see '\n", |
| 153 | + " 'if the exercise was done correctly.'.format(results, duration))" |
| 154 | + ] |
| 155 | + }, |
| 156 | + { |
| 157 | + "cell_type": "markdown", |
| 158 | + "metadata": {}, |
| 159 | + "source": [ |
| 160 | + "**VERIFY:** Run some checks to verify that the changes you made to the code were correct. Some of the checks should fail when you initially run the cells. After completing the exercises, the checks should pass." |
| 161 | + ] |
| 162 | + }, |
| 163 | + { |
| 164 | + "cell_type": "code", |
| 165 | + "execution_count": null, |
| 166 | + "metadata": {}, |
| 167 | + "outputs": [], |
| 168 | + "source": [ |
| 169 | + "assert results == [0, 1, 2, 3], 'Did you remember to call ray.get?'\n", |
| 170 | + "assert duration < 1.1, ('The loop took {} seconds. This is too slow.'\n", |
| 171 | + " .format(duration))\n", |
| 172 | + "assert duration > 1, ('The loop took {} seconds. This is too fast.'\n", |
| 173 | + " .format(duration))\n", |
| 174 | + "\n", |
| 175 | + "print('Success! The example took {} seconds.'.format(duration))" |
| 176 | + ] |
| 177 | + }, |
| 178 | + { |
| 179 | + "cell_type": "markdown", |
| 180 | + "metadata": {}, |
| 181 | + "source": [ |
| 182 | + "**EXERCISE:** Use the UI to view the task timeline and to verify that the four tasks were executed in parallel. You can do this as follows.\n", |
| 183 | + "\n", |
| 184 | + "1. Run the following cell to generate a JSON file containing the profiling data.\n", |
| 185 | + "2. Download the timeline file by right clicking on `timeline01.json` in the navigator to the left and choosing **\"Download\"**.\n", |
| 186 | + "3. Open [`chrome://tracing/`](chrome://tracing/) in the Chrome web browser, click on the **\"Load\"** button and load the downloaded JSON file.\n", |
| 187 | + "\n", |
| 188 | + "To navigate within the timeline, do the following.\n", |
| 189 | + "- Move around by clicking and dragging.\n", |
| 190 | + "- Zoom in and out by holding **alt** (or **option**) and scrolling.\n", |
| 191 | + "\n", |
| 192 | + "**NOTE:** The timeline visualization will only work in **Chrome**." |
| 193 | + ] |
| 194 | + }, |
| 195 | + { |
| 196 | + "cell_type": "code", |
| 197 | + "execution_count": null, |
| 198 | + "metadata": {}, |
| 199 | + "outputs": [], |
| 200 | + "source": [ |
| 201 | + "ray.timeline(filename=\"timeline01.json\")" |
| 202 | + ] |
| 203 | + } |
| 204 | + ], |
| 205 | + "metadata": { |
| 206 | + "kernelspec": { |
| 207 | + "display_name": "Python 3", |
| 208 | + "language": "python", |
| 209 | + "name": "python3" |
| 210 | + }, |
| 211 | + "language_info": { |
| 212 | + "codemirror_mode": { |
| 213 | + "name": "ipython", |
| 214 | + "version": 3 |
| 215 | + }, |
| 216 | + "file_extension": ".py", |
| 217 | + "mimetype": "text/x-python", |
| 218 | + "name": "python", |
| 219 | + "nbconvert_exporter": "python", |
| 220 | + "pygments_lexer": "ipython3", |
| 221 | + "version": "3.7.4" |
| 222 | + }, |
| 223 | + "toc": { |
| 224 | + "base_numbering": 1, |
| 225 | + "nav_menu": {}, |
| 226 | + "number_sections": false, |
| 227 | + "sideBar": true, |
| 228 | + "skip_h1_title": false, |
| 229 | + "title_cell": "Table of Contents", |
| 230 | + "title_sidebar": "Contents", |
| 231 | + "toc_cell": false, |
| 232 | + "toc_position": { |
| 233 | + "height": "calc(100% - 180px)", |
| 234 | + "left": "10px", |
| 235 | + "top": "150px", |
| 236 | + "width": "382.391px" |
| 237 | + }, |
| 238 | + "toc_section_display": true, |
| 239 | + "toc_window_display": true |
| 240 | + } |
| 241 | + }, |
| 242 | + "nbformat": 4, |
| 243 | + "nbformat_minor": 4 |
| 244 | +} |
0 commit comments