Skip to content

[Verify Schema] add step missing example notebook #956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Eyal-Danieli merged 1 commit into from
Dec 31, 2025
Merged

[Verify Schema] add step missing example notebook #956

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 226 additions & 8 deletions steps/src/verify_schema/verify_schema.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,249 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "cfd2162d-0f64-4a9d-a36b-01c05a05c6cc",
"metadata": {},
"source": [
"# Verify Schema Step Demo\n",
"This notebook walks through a simple example of using the verify-schema step in serving functions, by first downloading the step from the hub and inpecting it, then including the step in the function in two different approaches."
]
},
{
"cell_type": "markdown",
"id": "cc1eb162-f52b-4012-a5c8-07a2259bb7f7",
"metadata": {},
"source": [
"## Get the step from the hub"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d814bd50-4eac-4cb4-9eec-9f1b104df85e",
"metadata": {},
"outputs": [],
"source": [
"import mlrun"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "c6cd12de-1818-4e23-988e-6ede4837cd24",
"metadata": {},
"outputs": [],
"source": [
"hub_step = mlrun.get_hub_step(\"hub://verify_schema\")"
]
},
{
"cell_type": "markdown",
"id": "0a565753-b7da-409a-807d-d77e1ec98517",
"metadata": {},
"source": [
"Now the file has been downloaded to our current working directory (provide `local_path` paramater to change this default destination), and we can edit it directly. \n",
"\n",
"We can also inspect the `hub_stpe` object:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "37ba4e3b-e337-4651-ba60-69724b00d3eb",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'filename': 'verify_schema.py',\n",
" 'example': 'verify_schema.ipynb',\n",
" 'local_path': PosixPath('/User'),\n",
" 'url': 'hub://verify_schema',\n",
" 'class_name': 'VerifySchema',\n",
" 'name': 'verify_schema',\n",
" 'version': '1.0.0',\n",
" 'categories': ['data-preparation', 'model-serving', 'utilities'],\n",
" 'description': 'Verifies the event is aligned with the provided schema'}"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"hub_step.to_dict()"
]
},
{
"cell_type": "markdown",
"id": "8f702135-5dff-4355-bc5c-da9146b3eff7",
"metadata": {},
"source": [
"## Use custom step in a function"
]
},
{
"cell_type": "markdown",
"id": "73ed3007-f40f-4da8-a3a7-cd9933491322",
"metadata": {},
"source": [
"When we are done with editing the step code, we are ready to use it in a function, as any other custom step:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "9f26755e-5053-4103-9fcd-86bf839f6402",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"> 2025-12-31 09:00:01,987 [info] Project loaded successfully: {\"project_name\":\"verify-schema-step-demo\"}\n"
]
}
],
"source": [
"project = mlrun.get_or_create_project(\"verify-schema-step-demo\",'./verify-schema-step-demo')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "20a3cd67-b266-4b2d-9016-2139a2ab7f53",
"metadata": {},
"outputs": [],
"source": [
"fn = project.set_function(hub_step.get_src_file_path(), name=\"serving-fn\", kind=\"serving\", image = \"mlrun/mlrun\" )\n",
"graph = fn.set_topology(\"flow\", engine=\"async\")\n",
"schema = [\"id\", \"height\", \"weight\"]\n",
"graph.to(class_name=\"VerifySchema\", name=\"verify\", schema=schema)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "8b63efcf-22b0-482b-853c-a4991814693b",
"metadata": {},
"outputs": [],
"source": [
"project.deploy_function(fn)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "443a9138-e1b1-41e0-8a60-dc28f2680af9",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'id': '3425', 'height': 157, 'weight': 58}"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"serving_fn = project.get_function(\"serving-fn\")\n",
"event = {\n",
" \"id\": \"3425\",\n",
" \"height\": 157,\n",
" \"weight\": 58\n",
"}\n",
"serving_fn.invoke(\"/\", body=event)"
]
},
{
"cell_type": "markdown",
"id": "4933f6c7-6442-4756-812c-fd5a257ee540",
"metadata": {},
"source": [
"Or try to raise an error:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "98b8ff5a-783c-414c-bca7-eb5efe702b70",
"metadata": {},
"outputs": [],
"source": [
"event = {\n",
" \"id\": \"3425\",\n",
" \"height\": 157\n",
"}\n",
"serving_fn.invoke(\"/\", body=event)"
]
},
{
"cell_type": "markdown",
"id": "83182c10-223c-473d-a17a-2ddc94dd0821",
"metadata": {},
"source": [
"## Add step directly from the hub\n",
"In case you don't need to make any changes to the step code, you can add the step directly from the hub: "
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "7ce84cdc-8a32-4db9-aa73-171a622d1617",
"metadata": {},
"outputs": [],
"source": [
"graph.add_step(class_name=\"hub://verify_schema\", name=\"verify-2\", schema=schema).respond()"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "16ff979d-67e3-40c6-93de-aacd50b80fc1",
"metadata": {},
"outputs": [],
"source": [
"project.deploy_function(fn)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "bf83904c-7be3-4eaa-9564-386bd7ebe77b",
"metadata": {},
"outputs": [],
"execution_count": null,
"source": "",
"id": "556b36b9b89d0515"
"source": [
"event = {\n",
" \"id\": \"3425\",\n",
" \"height\": 157,\n",
" \"weight\": 58\n",
"}\n",
"serving_fn.invoke(\"/\", body=event)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "mlrun-base-py311",
"language": "python",
"name": "python3"
"name": "conda-env-mlrun-base-py311-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"pygments_lexer": "ipython3",
"version": "3.11.12"
}
},
"nbformat": 4,
Expand Down