Fix READMEs and ingored_files in cloud build triggers

Author: ddeleo

README.md

@@ -28,6 +28,7 @@ This repository is broken up into:
 *   [UDFs](/udfs) - User-defined functions for common usage as well as migration
     *   [community](/udfs/community) - Community contributed user-defined
         functions
+    *   [datasketches](/udfs/datasketches) - UDFs deployed from the latest release of [Apache Datasketches for BigQuery](https://github.com/apache/datasketches-bigquery)
     *   [migration](/udfs/migration) - UDFs which mimic the behavior of
         proprietary functions in the following databases:
         *   [netezza](/udfs/migration/netezza)

release/terraform/main.tf

@@ -50,7 +50,7 @@ resource "google_cloudbuild_trigger" "regional_trigger" {
     }
   }
   included_files = ["udfs/**", "stored_procedures/**"]
-  ignored_files = ["cloudbuild.yaml", ".*\\.md", "images/*", "tools/**", "udfs/datasketches/**"]
+  ignored_files = ["**/*.md"]
   include_build_logs = "INCLUDE_BUILD_LOGS_WITH_STATUS"
 
   substitutions = {

udfs/README.md

@@ -7,15 +7,21 @@ function using another SQL expression or JavaScript. These functions accept
 columns of input and perform actions, returning the result of those actions as a
 value.
 
-## Community and Migration Functions
-
-The [community](/udfs/community) folder contains community-contributed functions
-that perform some actions in BigQuery. The [migration](/udfs/migration) folder
-contains sub-folders such as [teradata](/udfs/migration/teradata),
-[redshift](/udfs/migration/redshift), [sqlserver](/udfs/migration/sqlserver) and [oracle](/udfs/migration/oracle) which
-contain community-contributed functions that replicate the behavior of
-proprietary functions in other data warehouses. These functions can help you
-achieve feature parity in a migration from another data warehouse to BigQuery.
+## Repo Folder to BigQuery Dataset Mappings
+
+> [!IMPORTANT]
+> The UDF datasets listed below reside in US multi-region, but are also available in all other supported BigQuery locations as described in the [Using the UDFs](#using-the-udfs) section.
+
+| Repo Folder                                         | BigQuery UDF Dataset                                                                                       | Description                                                                                                                                                                                                                                                                                                                            |
+|-----------------------------------------------------|------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [`community/`](/udfs/community)                     | [`bqutil.fn`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2sfn)                     | Contains an assortment of community-contributed functions.                                                                                                                                                                                                                                                                             |
+| [`datasketches/`](/udfs/datasketches/)              | [`bqutil.datasketches`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2sdatasketches) | Contains an open source, high-performance library of stochastic streaming algorithms commonly called "sketches". The source for these UDFs are maintained in the apache/datasketches-bigquery repo, but are available in the bqutil.datasketches US multi-region dataset and all other regions as described in Using the UDFs section. |
+| [`migration/oracle/`](/udfs/migration/oracle)       | [`bqutil.or`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2sor)                     | Contains community-contributed functions that replicate the behavior of Oracle functions.                                                                                                                                                                                                                                              |
+| [`migration/redshift/`](/udfs/migration/redshift)   | [`bqutil.rs`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2srs)                     | Contains community-contributed functions that replicate the behavior of Redshift functions.                                                                                                                                                                                                                                            |
+| [`migration/snowflake/`](/udfs/migration/snowflake) | [`bqutil.sf`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2ssf)                     | Contains community-contributed functions that replicate the behavior of Snowflake functions.                                                                                                                                                                                                                                           |
+| [`migration/sqlserver/`](/udfs/migration/sqlserver) | [`bqutil.ss`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2sss)                     | Contains community-contributed functions that replicate the behavior of SQL Server functions.                                                                                                                                                                                                                                          |
+| [`migration/teradata/`](/udfs/migration/teradata/)  | [`bqutil.td`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2std)                     | Contains community-contributed functions that replicate the behavior of Teradata functions.                                                                                                                                                                                                                                            |
+| [`migration/vertica/`](/udfs/migration/vertica)     | [`bqutil.ve`](https://console.cloud.google.com/bigquery?ws=!1m4!1m3!3m2!1sbqutil!2sve)                     | Contains community-contributed functions that replicate the behavior of Vertica functions.                                                                                                                                                                                                                                             |
 
 ## Using the UDFs
 
@@ -26,23 +32,20 @@ publicly shared datasets. Queries can then reference the shared UDFs in the US m
 UDFs within this repository are also deployed publicly into every other region that [BigQuery supports](https://cloud.google.com/bigquery/docs/locations). 
 In order to use a UDF in your desired location outside of the US multi-region, you can reference it via a dataset with a regional suffix:
 
-`bqutil.<dataset>_<region>.<function>()`.
+`bqutil.<dataset>_<region>.<function>()`
 
 For example, the Teradata `nullifzero` can be referenced in various locations:
 
-```
-bqutil.td_eu.nullifzero()            ## eu multi-region
-
-bqutil.td_europe_west1.nullifzero()  ## europe-west1 region
+```sql
+bqutil.td_eu.nullifzero()            -- eu multi-region
 
-bqutil.td_asia_south1.nullifzero()   ## asia-south1 region
+bqutil.td_europe_west1.nullifzero()  -- europe-west1 region
 
+bqutil.td_asia_south1.nullifzero()   -- asia-south1 region
 ```
 
-Note: Region suffixes added to dataset names replace `-` with `_` in order to comply with BigQuery dataset naming rules.
-
-
-![Alt text](/images/public_udf_architecture.png?raw=true "Public UDFs")
+> [!NOTE]  
+> Region suffixes added to dataset names replace `-` with `_` in order to comply with BigQuery dataset naming rules.
 
 ## Deploying the UDFs
 
@@ -60,6 +63,8 @@ The following sections cover a few methods of deploying the UDFs.
 
 ### Deploy with Cloud Build (Recommended)
 
+<details><summary><b>&#128466; Click to expand step-by-step instructions</b></summary>
+
 1. Authenticate using the Cloud SDK and set the BigQuery project in which you'll
    deploy your UDF(s):
 
@@ -86,13 +91,19 @@ The following sections cover a few methods of deploying the UDFs.
    # Deploy to US
    gcloud builds submit . --config=deploy.yaml --substitutions _PROJECT_ID=YOUR_PROJECT_ID,_BQ_LOCATION=US
    ```
-   > Note: Deploy to a different location by setting `_BQ_LOCATION` to your own
+
+   > IMPORTANT:
+   > Deploy to a different location by setting `_BQ_LOCATION` to your own
    > desired value.\
    > [Click here](https://cloud.google.com/bigquery/docs/locations#supported_regions)
    > for a list of supported locations.
 
+</details>
+
 ### Deploy with your own machine
 
+<details><summary><b>&#128466; Click to expand step-by-step instructions</b></summary>
+
 Run the following in your machine's terminal to deploy all UDFs in your own
 BigQuery project.
 
@@ -117,13 +128,19 @@ BigQuery project.
    # Deploy to US
    export BQ_LOCATION=US && bash deploy.sh
    ```
-   > Note: Deploy to a different location by setting `BQ_LOCATION` to your own
+
+   > IMPORTANT:
+   > Deploy to a different location by setting `BQ_LOCATION` to your own
    > desired value.\
    > [Click here](https://cloud.google.com/bigquery/docs/locations#supported_regions)
    > for a list of supported locations.
 
+</details>
+
 ### Deploy with bq command-line tool or BigQuery Console
 
+<details><summary><b>&#128466; Click to expand step-by-step instructions</b></summary>
+
 If you want to create the UDFs from this repository using the bq command-line
 tool, then you must make a few modifications to the SQLX files as shown below:
 
@@ -138,8 +155,12 @@ tool, then you must make a few modifications to the SQLX files as shown below:
       ```
     * BigQuery Console: Just paste the SQL UDF body in the console and execute.
 
+</details>
+
 ### Using JavaScript UDFs
 
+<details><summary><b>&#128466; Click to expand step-by-step instructions</b></summary>
+
 When
 creating [JavaScript UDFs](https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions#javascript-udf-structure)
 in your dataset, you need both to create the UDF and optionally copy the
@@ -153,7 +174,7 @@ UDF function, that uses the `js-levenshtein-v1.1.6.js` library.
 
 1. Copy the compiled library to your bucket:
    `gsutil cp gs://bqutil-lib/bq_js_libs/js-levenshtein-v1.1.6.js gs://your-bucket`
-2. Give permissions to the library. First, if you don't
+1. Give permissions to the library. First, if you don't
    have [uniform bucket-level access](https://cloud.google.com/storage/docs/using-uniform-bucket-level-access)
    in your bucket, enable
    it: `gsutil uniformbucketlevelaccess set on gs://your-bucket`. Once done give
@@ -163,13 +184,17 @@ UDF function, that uses the `js-levenshtein-v1.1.6.js` library.
    or [project](https://cloud.google.com/sdk/gcloud/reference/projects/add-iam-policy-binding)
    level to a user or
    group: `gsutil iam ch [user|group]:[user|group]@domain.com:roles/storage.objectViewer gs://your_bucket`
-3. Edit the [levenshtein.sql](community/levenshtein.sql) SQL file and replace
+1. Edit the [levenshtein.sql](community/levenshtein.sql) SQL file and replace
    the library path `library="${JS_BUCKET}/js-levenshtein-v1.1.6.js"` with your
    own path `library="gs://your-bucket/js-levenshtein-v1.1.6.js`
-4. Create the SQL UDF passing the previously modified SQL file:
+1. Create the SQL UDF passing the previously modified SQL file:
    `bq query --project_id YOUR_PROJECT_ID --dataset_id YOUR_DATASET_ID --nouse_legacy_sql < levenshtein.sql`
 
+</details>
+
 ## Contributing UDFs
 
+![Alt text](/images/public_udf_architecture.png?raw=true "Public UDFs")
+
 If you are interested in contributing UDFs to this repository, please see the
 [instructions](/udfs/CONTRIBUTING.md) to get started.