Skip to content

Commit 2a9ac72

Browse files
dimitri-yatsenkodimitri-yatsenko
committed
cleanup setup/connect.
1 parent f016bac commit 2a9ac72

File tree

1 file changed

+140
-67
lines changed

1 file changed

+140
-67
lines changed

‎book/10-setup/02-connect.ipynb‎

Lines changed: 140 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,65 @@
1212
"---"
1313
]
1414
},
15+
{
16+
"cell_type": "markdown",
17+
"metadata": {},
18+
"source": [
19+
"\n",
20+
"This textbook comes with a Devcontainer that runs a MySQL database server with the following credentials:\n",
21+
"- database server address: `db`\n",
22+
"- username: `dev`\n",
23+
"- password: `devpass`"
24+
]
25+
},
26+
{
27+
"cell_type": "markdown",
28+
"metadata": {},
29+
"source": [
30+
"# Connect with SQL \"Magic\" for Jupyter Notebooks\n",
31+
"\n",
32+
"You can execute SQL statements directly from Jupyter with the help of SQL \"Jupyter magic\" implemented by the [`sqljupy`](https://ploomber.io/blog/jupysql/) library.\n",
33+
"\n",
34+
"\n",
35+
"The following cell sets up the connection to the database for the Jupyter SQL Magic."
36+
]
37+
},
38+
{
39+
"cell_type": "markdown",
40+
"metadata": {},
41+
"source": []
42+
},
43+
{
44+
"cell_type": "code",
45+
"execution_count": null,
46+
"metadata": {},
47+
"outputs": [
48+
{
49+
"name": "stdout",
50+
"output_type": "stream",
51+
"text": [
52+
"The sql extension is already loaded. To reload it, use:\n",
53+
" %reload_ext sql\n"
54+
]
55+
},
56+
{
57+
"data": {
58+
"text/html": [
59+
"<span style=\"None\">Connecting to &#x27;mysql+pymysql://dev:***@db&#x27;</span>"
60+
],
61+
"text/plain": [
62+
"Connecting to 'mysql+pymysql://dev:***@db'"
63+
]
64+
},
65+
"metadata": {},
66+
"output_type": "display_data"
67+
}
68+
],
69+
"source": [
70+
"%load_ext sql\n",
71+
"%sql mysql+pymysql://dev:devpass@db"
72+
]
73+
},
1574
{
1675
"cell_type": "markdown",
1776
"metadata": {},
@@ -20,7 +79,7 @@
2079
"\n",
2180
"The Devcontainer that comes with this textbook contains a running MySQL server.\n",
2281
"\n",
23-
"The root credentials are set in environment variables `DJ_HOST`, `DJ_USER`, and `DJ_PASS`.\n",
82+
"The user credentials are set in environment variables `DJ_HOST`, `DJ_USER`, and `DJ_PASS`.\n",
2483
"These credentials are not secret since this database is not exposed to the external world.\n",
2584
"\n",
2685
"The DataJoint client library uses these environment variables to connect to the database.\n",
@@ -31,16 +90,23 @@
3190
},
3291
{
3392
"cell_type": "code",
34-
"execution_count": 5,
93+
"execution_count": 1,
3594
"metadata": {},
3695
"outputs": [
96+
{
97+
"name": "stderr",
98+
"output_type": "stream",
99+
"text": [
100+
"[2025-09-16 01:37:46,977][INFO]: DataJoint 0.14.6 connected to dev@db:3306\n"
101+
]
102+
},
37103
{
38104
"data": {
39105
"text/plain": [
40-
"DataJoint connection (connected) root@localhost:3306"
106+
"DataJoint connection (connected) dev@db:3306"
41107
]
42108
},
43-
"execution_count": 5,
109+
"execution_count": 1,
44110
"metadata": {},
45111
"output_type": "execute_result"
46112
}
@@ -66,48 +132,6 @@
66132
"If you are reading this text to also learn SQL, you can use two ways to issue queries: with IPython magic commands or a client library."
67133
]
68134
},
69-
{
70-
"cell_type": "markdown",
71-
"metadata": {},
72-
"source": [
73-
"If you are only learning DataJoint, you are done.\n",
74-
"If you are reading this text to also learn SQL, you can use two ways to issue queries: with IPython magic commands or a client library."
75-
]
76-
},
77-
{
78-
"cell_type": "markdown",
79-
"metadata": {},
80-
"source": [
81-
"# Connect with IPython \"Magic\"\n",
82-
"\n",
83-
"You can execute SQL statements directly from Jupyter with the help of [\"magic commdands\"](https://towardsdatascience.com/jupyter-magics-with-sql-921370099589). \n",
84-
"\n",
85-
"The following cell sets up the connection to the database for the Jupyter SQL Magic."
86-
]
87-
},
88-
{
89-
"cell_type": "code",
90-
"execution_count": null,
91-
"metadata": {},
92-
"outputs": [],
93-
"source": [
94-
"import pymysql\n",
95-
"import os\n",
96-
"pymysql.install_as_MySQLdb()\n",
97-
"\n",
98-
"connection_string = \"mysql://{user}:{password}@{host}\".format(\n",
99-
" user=os.environ['DJ_USER'],\n",
100-
" host=os.environ['DJ_HOST'],\n",
101-
" password=os.environ['DJ_PASS']\n",
102-
")\n",
103-
"\n",
104-
"%load_ext sql\n",
105-
"%sql $connection_string\n",
106-
"\n",
107-
"%config SqlMagic.style = '_DEPRECATED_DEFAULT' # addresses a bug in the SQL magic extension\n",
108-
"\n"
109-
]
110-
},
111135
{
112136
"cell_type": "markdown",
113137
"metadata": {},
@@ -118,35 +142,84 @@
118142
},
119143
{
120144
"cell_type": "code",
121-
"execution_count": 4,
145+
"execution_count": 3,
122146
"metadata": {
123147
"vscode": {
124148
"languageId": "sql"
125149
}
126150
},
127151
"outputs": [
128152
{
129-
"name": "stdout",
130-
"output_type": "stream",
131-
"text": [
132-
" * mysql://root:***@localhost\n",
133-
"5 rows affected.\n"
134-
]
153+
"data": {
154+
"text/html": [
155+
"<span style=\"None\">Running query in &#x27;mysql+pymysql://dev:***@db&#x27;</span>"
156+
],
157+
"text/plain": [
158+
"Running query in 'mysql+pymysql://dev:***@db'"
159+
]
160+
},
161+
"metadata": {},
162+
"output_type": "display_data"
135163
},
136164
{
137-
"ename": "KeyError",
138-
"evalue": "'DEFAULT'",
139-
"output_type": "error",
140-
"traceback": [
141-
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
142-
"\u001b[31mKeyError\u001b[39m Traceback (most recent call last)",
143-
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m \u001b[43mget_ipython\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrun_cell_magic\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43msql\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43m-- show all users\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[33;43mSELECT User FROM mysql.user\u001b[39;49m\u001b[38;5;130;43;01m\\n\u001b[39;49;00m\u001b[33;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n",
144-
"\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2565\u001b[39m, in \u001b[36mInteractiveShell.run_cell_magic\u001b[39m\u001b[34m(self, magic_name, line, cell)\u001b[39m\n\u001b[32m 2563\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m.builtin_trap:\n\u001b[32m 2564\u001b[39m args = (magic_arg_s, cell)\n\u001b[32m-> \u001b[39m\u001b[32m2565\u001b[39m result = \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 2567\u001b[39m \u001b[38;5;66;03m# The code below prevents the output from being displayed\u001b[39;00m\n\u001b[32m 2568\u001b[39m \u001b[38;5;66;03m# when using magics with decorator @output_can_be_silenced\u001b[39;00m\n\u001b[32m 2569\u001b[39m \u001b[38;5;66;03m# when the last Python token in the expression is a ';'.\u001b[39;00m\n\u001b[32m 2570\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, \u001b[38;5;28;01mFalse\u001b[39;00m):\n",
145-
"\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/lib/python3.11/site-packages/sql/magic.py:219\u001b[39m, in \u001b[36mSqlMagic.execute\u001b[39m\u001b[34m(self, line, cell, local_ns)\u001b[39m\n\u001b[32m 216\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[32m 218\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m219\u001b[39m result = \u001b[43msql\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrun\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrun\u001b[49m\u001b[43m(\u001b[49m\u001b[43mconn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparsed\u001b[49m\u001b[43m[\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43msql\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muser_ns\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 221\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (\n\u001b[32m 222\u001b[39m result \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 223\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(result, \u001b[38;5;28mstr\u001b[39m)\n\u001b[32m (...)\u001b[39m\u001b[32m 226\u001b[39m \u001b[38;5;66;03m# Instead of returning values, set variables directly in the\u001b[39;00m\n\u001b[32m 227\u001b[39m \u001b[38;5;66;03m# user's namespace. Variable names given by column names\u001b[39;00m\n\u001b[32m 229\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.autopandas:\n",
146-
"\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/lib/python3.11/site-packages/sql/run.py:374\u001b[39m, in \u001b[36mrun\u001b[39m\u001b[34m(conn, sql, config, user_namespace)\u001b[39m\n\u001b[32m 372\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m result \u001b[38;5;129;01mand\u001b[39;00m config.feedback:\n\u001b[32m 373\u001b[39m \u001b[38;5;28mprint\u001b[39m(interpret_rowcount(result.rowcount))\n\u001b[32m--> \u001b[39m\u001b[32m374\u001b[39m resultset = \u001b[43mResultSet\u001b[49m\u001b[43m(\u001b[49m\u001b[43mresult\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 375\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m config.autopandas:\n\u001b[32m 376\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m resultset.DataFrame()\n",
147-
"\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/lib/python3.11/site-packages/sql/run.py:116\u001b[39m, in \u001b[36mResultSet.__init__\u001b[39m\u001b[34m(self, sqlaproxy, config)\u001b[39m\n\u001b[32m 114\u001b[39m \u001b[38;5;28mlist\u001b[39m.\u001b[34m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, sqlaproxy.fetchall())\n\u001b[32m 115\u001b[39m \u001b[38;5;28mself\u001b[39m.field_names = unduplicate_field_names(\u001b[38;5;28mself\u001b[39m.keys)\n\u001b[32m--> \u001b[39m\u001b[32m116\u001b[39m \u001b[38;5;28mself\u001b[39m.pretty = PrettyTable(\u001b[38;5;28mself\u001b[39m.field_names, style=\u001b[43mprettytable\u001b[49m\u001b[43m.\u001b[49m\u001b[34;43m__dict__\u001b[39;49m\u001b[43m[\u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m.\u001b[49m\u001b[43mstyle\u001b[49m\u001b[43m.\u001b[49m\u001b[43mupper\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m]\u001b[49m)\n\u001b[32m 117\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 118\u001b[39m \u001b[38;5;28mlist\u001b[39m.\u001b[34m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, [])\n",
148-
"\u001b[31mKeyError\u001b[39m: 'DEFAULT'"
149-
]
165+
"data": {
166+
"text/html": [
167+
"<span style=\"color: green\">6 rows affected.</span>"
168+
],
169+
"text/plain": [
170+
"6 rows affected."
171+
]
172+
},
173+
"metadata": {},
174+
"output_type": "display_data"
175+
},
176+
{
177+
"data": {
178+
"text/html": [
179+
"<table>\n",
180+
" <thead>\n",
181+
" <tr>\n",
182+
" <th>User</th>\n",
183+
" </tr>\n",
184+
" </thead>\n",
185+
" <tbody>\n",
186+
" <tr>\n",
187+
" <td>dev</td>\n",
188+
" </tr>\n",
189+
" <tr>\n",
190+
" <td>root</td>\n",
191+
" </tr>\n",
192+
" <tr>\n",
193+
" <td>mysql.infoschema</td>\n",
194+
" </tr>\n",
195+
" <tr>\n",
196+
" <td>mysql.session</td>\n",
197+
" </tr>\n",
198+
" <tr>\n",
199+
" <td>mysql.sys</td>\n",
200+
" </tr>\n",
201+
" <tr>\n",
202+
" <td>root</td>\n",
203+
" </tr>\n",
204+
" </tbody>\n",
205+
"</table>"
206+
],
207+
"text/plain": [
208+
"+------------------+\n",
209+
"| User |\n",
210+
"+------------------+\n",
211+
"| dev |\n",
212+
"| root |\n",
213+
"| mysql.infoschema |\n",
214+
"| mysql.session |\n",
215+
"| mysql.sys |\n",
216+
"| root |\n",
217+
"+------------------+"
218+
]
219+
},
220+
"execution_count": 3,
221+
"metadata": {},
222+
"output_type": "execute_result"
150223
}
151224
],
152225
"source": [
@@ -236,7 +309,7 @@
236309
"name": "python",
237310
"nbconvert_exporter": "python",
238311
"pygments_lexer": "ipython3",
239-
"version": "3.11.13"
312+
"version": "3.13.2"
240313
}
241314
},
242315
"nbformat": 4,

0 commit comments

Comments
 (0)