|
109 | 109 | "id": "3Sq3sGfdt89E" |
110 | 110 | }, |
111 | 111 | "source": [ |
112 | | - "## Before you begin\r\n", |
113 | | - "\r\n", |
114 | | - "### GPU run-time\r\n", |
115 | | - "\r\n", |
116 | | - "*Make sure you're running this notebook in a GPU runtime if you have that option. In Colab, select* **Runtime > Change Runtime Type > GPU**\r\n", |
117 | | - "\r\n", |
118 | | - "### Set up your GCP project\r\n", |
119 | | - "\r\n", |
120 | | - "**The following steps are required, regardless of your notebook environment.**\r\n", |
121 | | - "\r\n", |
122 | | - "1. [Select or create a GCP project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\r\n", |
123 | | - "\r\n", |
124 | | - "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\r\n", |
125 | | - "\r\n", |
126 | | - "3. [Enable the Vertex APIs and Compute Engine APIs.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component)\r\n", |
127 | | - "\r\n", |
128 | | - "4. [Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Google Cloud Notebooks.\r\n", |
129 | | - "\r\n", |
130 | | - "5. Enter your project ID in the cell below. Then run the cell to make sure the\r\n", |
131 | | - "Cloud SDK uses the right project for all the commands in this notebook.\r\n", |
132 | | - "\r\n", |
| 112 | + "## Before you begin\n", |
| 113 | + "\n", |
| 114 | + "### GPU run-time\n", |
| 115 | + "\n", |
| 116 | + "*Make sure you're running this notebook in a GPU runtime if you have that option. In Colab, select* **Runtime > Change Runtime Type > GPU**\n", |
| 117 | + "\n", |
| 118 | + "### Set up your GCP project\n", |
| 119 | + "\n", |
| 120 | + "**The following steps are required, regardless of your notebook environment.**\n", |
| 121 | + "\n", |
| 122 | + "1. [Select or create a GCP project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n", |
| 123 | + "\n", |
| 124 | + "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n", |
| 125 | + "\n", |
| 126 | + "3. [Enable the Vertex APIs and Compute Engine APIs.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component)\n", |
| 127 | + "\n", |
| 128 | + "4. [Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Google Cloud Notebooks.\n", |
| 129 | + "\n", |
| 130 | + "5. Enter your project ID in the cell below. Then run the cell to make sure the\n", |
| 131 | + "Cloud SDK uses the right project for all the commands in this notebook.\n", |
| 132 | + "\n", |
133 | 133 | "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands." |
134 | 134 | ] |
135 | 135 | }, |
|
240 | 240 | "id": "9zpjPUOhvRQz" |
241 | 241 | }, |
242 | 242 | "source": [ |
243 | | - "### Authenticate your GCP account\r\n", |
244 | | - "\r\n", |
245 | | - "**If you are using Google Cloud Notebooks**, your environment is already\r\n", |
246 | | - "authenticated. Skip this step.\r\n", |
247 | | - "\r\n", |
| 243 | + "### Authenticate your GCP account\n", |
| 244 | + "\n", |
| 245 | + "**If you are using Google Cloud Notebooks**, your environment is already\n", |
| 246 | + "authenticated. Skip this step.\n", |
| 247 | + "\n", |
248 | 248 | "*Note: If you are on an Vertex notebook and run the cell, the cell knows to skip executing the authentication steps.*" |
249 | 249 | ] |
250 | 250 | }, |
|
1459 | 1459 | "id": "gM-YixlLmDy9" |
1460 | 1460 | }, |
1461 | 1461 | "source": [ |
1462 | | - "### Make a batch prediction file\r\n", |
1463 | | - "\r\n" |
| 1462 | + "### Make a batch prediction file\n" |
1464 | 1463 | ] |
1465 | 1464 | }, |
1466 | 1465 | { |
|
1489 | 1488 | "! gsutil cat $gcs_test_item" |
1490 | 1489 | ] |
1491 | 1490 | }, |
1492 | | - { |
1493 | | - "cell_type": "markdown", |
1494 | | - "metadata": { |
1495 | | - "id": "sPupiwqN_jAB" |
1496 | | - }, |
1497 | | - "source": [ |
1498 | | - "*Example output*:\n", |
1499 | | - "```\n", |
1500 | | - "{\"content\": \"gs://migration-ucaip-trainingaip-20210301154552/test.txt\", \"mime_type\": \"text/plain\"}\n", |
1501 | | - "Molecular basis of hexosaminidase A deficiency and pseudodeficiency in the Berks County Pennsylvania Dutch.\\tFollowing the birth of two infants with Tay-Sachs disease ( TSD ) , a non-Jewish , Pennsylvania Dutch kindred was screened for TSD carriers using the biochemical assay . A high frequency of individuals who appeared to be TSD heterozygotes was detected ( Kelly et al . , 1975 ) . Clinical and biochemical evidence suggested that the increased carrier frequency was due to at least two altered alleles for the hexosaminidase A alpha-subunit . We now report two mutant alleles in this Pennsylvania Dutch kindred , and one polymorphism . One allele , reported originally in a French TSD patient ( Akli et al . , 1991 ) , is a GT-- > AT transition at the donor splice-site of intron 9 . The second , a C-- > T transition at nucleotide 739 ( Arg247Trp ) , has been shown by Triggs-Raine et al . ( 1992 ) to be a clinically benign \" pseudodeficient \" allele associated with reduced enzyme activity against artificial substrate . Finally , a polymorphism [ G-- > A ( 759 ) ] , which leaves valine at codon 253 unchanged , is described\n", |
1502 | | - "```\n" |
1503 | | - ] |
1504 | | - }, |
1505 | 1491 | { |
1506 | 1492 | "cell_type": "markdown", |
1507 | 1493 | "metadata": { |
|
1557 | 1543 | ")" |
1558 | 1544 | ] |
1559 | 1545 | }, |
1560 | | - { |
1561 | | - "cell_type": "markdown", |
1562 | | - "metadata": { |
1563 | | - "id": "sPupiwqN_jAB" |
1564 | | - }, |
1565 | | - "source": [ |
1566 | | - "*Example output*:\n", |
1567 | | - "```\n", |
1568 | | - "{\n", |
1569 | | - " \"parent\": \"projects/migration-ucaip-training/locations/us-central1\",\n", |
1570 | | - " \"batchPredictionJob\": {\n", |
1571 | | - " \"displayName\": \"ten_20210301154552\",\n", |
1572 | | - " \"model\": \"projects/116273516712/locations/us-central1/models/4400738115568795648\",\n", |
1573 | | - " \"inputConfig\": {\n", |
1574 | | - " \"instancesFormat\": \"jsonl\",\n", |
1575 | | - " \"gcsSource\": {\n", |
1576 | | - " \"uris\": [\n", |
1577 | | - " \"gs://migration-ucaip-trainingaip-20210301154552/test.jsonl\"\n", |
1578 | | - " ]\n", |
1579 | | - " }\n", |
1580 | | - " },\n", |
1581 | | - " \"outputConfig\": {\n", |
1582 | | - " \"predictionsFormat\": \"jsonl\",\n", |
1583 | | - " \"gcsDestination\": {\n", |
1584 | | - " \"outputUriPrefix\": \"gs://migration-ucaip-trainingaip-20210301154552/batch_output/\"\n", |
1585 | | - " }\n", |
1586 | | - " },\n", |
1587 | | - " \"dedicatedResources\": {\n", |
1588 | | - " \"machineSpec\": {\n", |
1589 | | - " \"machineType\": \"n1-standard-2\"\n", |
1590 | | - " },\n", |
1591 | | - " \"startingReplicaCount\": 1,\n", |
1592 | | - " \"maxReplicaCount\": 1\n", |
1593 | | - " }\n", |
1594 | | - " }\n", |
1595 | | - "}\n", |
1596 | | - "```\n" |
1597 | | - ] |
1598 | | - }, |
1599 | 1546 | { |
1600 | 1547 | "cell_type": "markdown", |
1601 | 1548 | "metadata": { |
|
1638 | 1585 | "print(MessageToJson(request.__dict__[\"_pb\"]))" |
1639 | 1586 | ] |
1640 | 1587 | }, |
1641 | | - { |
1642 | | - "cell_type": "markdown", |
1643 | | - "metadata": { |
1644 | | - "id": "sPupiwqN_jAB" |
1645 | | - }, |
1646 | | - "source": [ |
1647 | | - "*Example output*:\n", |
1648 | | - "```\n", |
1649 | | - "{\n", |
1650 | | - " \"name\": \"projects/116273516712/locations/us-central1/batchPredictionJobs/3588251799200464896\",\n", |
1651 | | - " \"displayName\": \"ten_20210301154552\",\n", |
1652 | | - " \"model\": \"projects/116273516712/locations/us-central1/models/4400738115568795648\",\n", |
1653 | | - " \"inputConfig\": {\n", |
1654 | | - " \"instancesFormat\": \"jsonl\",\n", |
1655 | | - " \"gcsSource\": {\n", |
1656 | | - " \"uris\": [\n", |
1657 | | - " \"gs://migration-ucaip-trainingaip-20210301154552/test.jsonl\"\n", |
1658 | | - " ]\n", |
1659 | | - " }\n", |
1660 | | - " },\n", |
1661 | | - " \"outputConfig\": {\n", |
1662 | | - " \"predictionsFormat\": \"jsonl\",\n", |
1663 | | - " \"gcsDestination\": {\n", |
1664 | | - " \"outputUriPrefix\": \"gs://migration-ucaip-trainingaip-20210301154552/batch_output/\"\n", |
1665 | | - " }\n", |
1666 | | - " },\n", |
1667 | | - " \"state\": \"JOB_STATE_PENDING\",\n", |
1668 | | - " \"completionStats\": {\n", |
1669 | | - " \"incompleteCount\": \"-1\"\n", |
1670 | | - " },\n", |
1671 | | - " \"createTime\": \"2021-03-01T17:59:42.777083Z\",\n", |
1672 | | - " \"updateTime\": \"2021-03-01T17:59:42.777083Z\"\n", |
1673 | | - "}\n", |
1674 | | - "```\n" |
1675 | | - ] |
1676 | | - }, |
1677 | 1588 | { |
1678 | 1589 | "cell_type": "code", |
1679 | 1590 | "execution_count": null, |
|
1739 | 1650 | "print(MessageToJson(request.__dict__[\"_pb\"]))" |
1740 | 1651 | ] |
1741 | 1652 | }, |
1742 | | - { |
1743 | | - "cell_type": "markdown", |
1744 | | - "metadata": { |
1745 | | - "id": "sPupiwqN_jAB" |
1746 | | - }, |
1747 | | - "source": [ |
1748 | | - "*Example output*:\n", |
1749 | | - "```\n", |
1750 | | - "{\n", |
1751 | | - " \"name\": \"projects/116273516712/locations/us-central1/batchPredictionJobs/3588251799200464896\",\n", |
1752 | | - " \"displayName\": \"ten_20210301154552\",\n", |
1753 | | - " \"model\": \"projects/116273516712/locations/us-central1/models/4400738115568795648\",\n", |
1754 | | - " \"inputConfig\": {\n", |
1755 | | - " \"instancesFormat\": \"jsonl\",\n", |
1756 | | - " \"gcsSource\": {\n", |
1757 | | - " \"uris\": [\n", |
1758 | | - " \"gs://migration-ucaip-trainingaip-20210301154552/test.jsonl\"\n", |
1759 | | - " ]\n", |
1760 | | - " }\n", |
1761 | | - " },\n", |
1762 | | - " \"outputConfig\": {\n", |
1763 | | - " \"predictionsFormat\": \"jsonl\",\n", |
1764 | | - " \"gcsDestination\": {\n", |
1765 | | - " \"outputUriPrefix\": \"gs://migration-ucaip-trainingaip-20210301154552/batch_output/\"\n", |
1766 | | - " }\n", |
1767 | | - " },\n", |
1768 | | - " \"state\": \"JOB_STATE_PENDING\",\n", |
1769 | | - " \"completionStats\": {\n", |
1770 | | - " \"incompleteCount\": \"-1\"\n", |
1771 | | - " },\n", |
1772 | | - " \"createTime\": \"2021-03-01T17:59:42.777083Z\",\n", |
1773 | | - " \"updateTime\": \"2021-03-01T17:59:42.777083Z\"\n", |
1774 | | - "}\n", |
1775 | | - "```\n" |
1776 | | - ] |
1777 | | - }, |
1778 | 1653 | { |
1779 | 1654 | "cell_type": "code", |
1780 | 1655 | "execution_count": null, |
|
1798 | 1673 | " time.sleep(60)" |
1799 | 1674 | ] |
1800 | 1675 | }, |
1801 | | - { |
1802 | | - "cell_type": "markdown", |
1803 | | - "metadata": { |
1804 | | - "id": "trainingpipelines_create:migration,new,response,icn" |
1805 | | - }, |
1806 | | - "source": [ |
1807 | | - "*Example output*:\n", |
1808 | | - "```\n", |
1809 | | - "gs://migration-ucaip-trainingaip-20210301154552/batch_output/prediction-ten_20210301154552-2021-03-01T17:59:42.638222Z/predictions_00001.jsonl\n", |
1810 | | - "{\"instance\":{\"content\":\"gs://migration-ucaip-trainingaip-20210301154552/test.txt\",\"mimeType\":\"text/plain\"},\"prediction\":{\"ids\":[\"7806436899697983488\",\"7806436899697983488\",\"7806436899697983488\",\"4347672385877442560\",\"4347672385877442560\",\"4347672385877442560\"],\"displayNames\":[\"SpecificDisease\",\"SpecificDisease\",\"SpecificDisease\",\"Modifier\",\"Modifier\",\"Modifier\"],\"textSegmentStartOffsets\":[\"149\",\"19\",\"169\",\"236\",\"688\",\"330\"],\"textSegmentEndOffsets\":[\"165\",\"45\",\"171\",\"238\",\"690\",\"332\"],\"confidences\":[0.99957836,0.9995628,0.9995044,0.9993287,0.9993144,0.99927235]}}\n", |
1811 | | - "```\n" |
1812 | | - ] |
1813 | | - }, |
1814 | 1676 | { |
1815 | 1677 | "cell_type": "markdown", |
1816 | 1678 | "metadata": { |
|
2338 | 2200 | "id": "bQ-VVaSxJjkd" |
2339 | 2201 | }, |
2340 | 2202 | "source": [ |
2341 | | - "# Cleaning up\r\n", |
2342 | | - "\r\n", |
2343 | | - "To clean up all GCP resources used in this project, you can [delete the GCP\r\n", |
2344 | | - "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\r\n", |
2345 | | - "\r\n", |
| 2203 | + "# Cleaning up\n", |
| 2204 | + "\n", |
| 2205 | + "To clean up all GCP resources used in this project, you can [delete the GCP\n", |
| 2206 | + "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n", |
| 2207 | + "\n", |
2346 | 2208 | "Otherwise, you can delete the individual resources you created in this tutorial." |
2347 | 2209 | ] |
2348 | 2210 | }, |
|
2404 | 2266 | ], |
2405 | 2267 | "metadata": { |
2406 | 2268 | "colab": { |
2407 | | - "name": "UJ7 unified AutoML for natural language with Vertex AI Text Entity Extraction.ipynb", |
| 2269 | + "name": "UJ7 AutoML for natural language with Vertex AI Text Entity Extraction.ipynb", |
2408 | 2270 | "toc_visible": true |
2409 | 2271 | }, |
2410 | 2272 | "kernelspec": { |
|
0 commit comments