[PP-2216] Update API doc for OAuthU2M Support

Author: jackyhu-db

ApiSpecifications.md

@@ -160,7 +160,8 @@ POST <partners/databricks/v1/connect>: [example, can be customized]
   "is_sql_warehouse": true|false, [optional: set if cluster_id is set. Determines whether cluster_id refers to Interactive Cluster or SQL Warehouse]
   "data_source_connector": "Oracle", [optional, unused and reserved for future use: for data connector tools, the name of the data source that the user should be referred to in their tool]
   "service_principal_id": "a2a25a05-3d59-4515-a73b-b8bc5ab79e31", [optional, the UUID (username) of the service principal identity]
-  "service_principal_oauth_secret": "dose..." [optional, the OAuth secret of the service principal identity, it will be passed only when partner config includes OAuth M2M auth option]
+  "service_principal_oauth_secret": "dose...", [optional, the OAuth secret of the service principal identity, it will be passed only when the partner config includes OAuth M2M auth option]
+  "oauth_u2m_app_id": "782b7906-20c4-4c12-8850-b26b77d125f5" [optional, the client ID of Databricks OAuth U2M app connection created by Partner Connect. It will be passed only when the partner config includes OAuth U2M auth option]
 }
 ```
 
@@ -174,7 +175,8 @@ Status Code: 200
   "connection_id": "7f2e4c43-9714-47cf-9011-d8148eaa27a2", [example, optional, see below]
   "user_status": "new", [example]
   "account_status": "existing", [example]
-  "configured_resources": true|false
+  "configured_resources": true|false,
+  "oauth_redirect_uri": "http://www.partner.com/oauth/callback [example, optional, see below]
  }
 ```
 Return values:
@@ -190,6 +192,7 @@ Return values:
 6. **configured\_resources** - a boolean that represents whether the partner configured/persisted the Databricks resources on this Connect API request.
     1. If **is\_connection\_established** is true, **configured\_resources** must be set, but will be ignored.
     2. If **is\_connection\_established** is false and **configured\_resources** is false, Databricks will delete the resources it provisioned.
+7. **oauth\_redirect\_uri** - the partner application's URL that handles Databricks OAuth redirect request in the OAuth U2M flow (Authorization code flow). It should be set only when the partner is configured with OAuth U2M as the auth option ([ParterConfig](./api-doc/Models/PartnerConfig.md) `auth_options` contains `AUTH_OAUTH_U2M`) and does not have a pre-registered Databricks published OAuth app connection ([ParterConfig](./api-doc/Models/PartnerConfig.md) `is_published_app` is `false` or `null`).
 
 **Failure Responses:**
 

OnboardingDoc.md

@@ -32,6 +32,10 @@ The following phrases will help you understand the Databricks product and this d
 - **Personal Access Token (PAT):** A token that a partner product can use to authenticate with Databricks
 - **Service Principal:** An account that a partner product can use when calling Databricks APIs. Service Principals have access controls associated with them.
 - **OAuth M2M** It uses service principals to authenticate Databricks. It is also known as 2-legged OAuth and OAuth Client Credentials Flow. Partner product can use service principal UUD (client_id) and OAuth secret (client_secret) to authenticate with Databricks.
+- **OAuth U2M** It allows users to access Databricks account and workspace resources via the partner application on behalf of a user. It is also known as 3-legged OAuth and OAuth Authorization Code Flow.
+- **Published OAuth application** The application that is pre-registered by Databricks and is available in all the Databricks accounts. For the app to be eligible for published application, it must meet the following requirements:
+  - The OAuth redirect URLs of the app should be fixed instead of account/customer/tenant specific
+  - The app cannot be confidential app (no client secret) and has to support [PKCE](https://oauth.net/2/pkce/)
 - **Service Principal OAuth Secret**: The service principal's secret that a partner product use it along with service principal UUID to authenticate with Databricks.
 
 

api-doc/Models/ConnectRequest.md

@@ -26,6 +26,7 @@
 | **service\_principal\_id** | **String** | The UUID (username) of the service principal identity that a partner product can use to call Databricks APIs. Note the format is different from the databricks_user_id field in user_info. If empty, no service principal was created | [optional] [default to null] |
 | **service\_principal\_oauth\_secret** | **String** | The OAuth secret of the service principal identity that a partner product can use to call Databricks APIs (see [OAuth M2M](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html)). It will be set only when the `auth_options` in the [PartnerConfig](PartnerConfig.md) contains the value `AUTH_OAUTH_M2M`. | [optional] [default to null] |
 | **connection\_scope** | **String** | The scope of users that can use this connection. Workspace means all users in the same workspace. User means only the user creating it. | [optional] [default to null] |
+| **oauth\_u2m\_app\_id** | **String** | The client ID of OAuth U2M app connection (created by Partner Connect) that a partner product can use to initiate Databricks OAuth U2M flow. It will be set only when the `auth_options` in the [PartnerConfig](PartnerConfig.md) contains the value `AUTH_OAUTH_U2M`. | [optional] [default to null] |
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
 

api-doc/Models/PartnerConfig.md

@@ -23,8 +23,9 @@
 | **require\_manual\_signup** | **Boolean** | True if the partner requires a manual signup after connect api is called. When set to true, connect api call with is_connection_established (sign in) is expected to return 404 account_not_found or connection_not_found until the user completes the manual signup step. | [optional] [default to null] |
 | **trial\_type** | **String** | Enum describing the type of trials the partner support. Partners can chose to support trial account expiration at the individual user or account level. If trial level is user, expiring one user connection should not expire another user in the same account. | [optional] [default to null] |
 | **supports\_demo** | **Boolean** | True if partner supports the demo flag in the connect api call. | [optional] [default to null] |
-| **auth\_options** | **List** | The available authentication methods that a partner can use to authenticate with Databricks. If it is not specified, `AUTH_PAT` will be used. The allowed options include <ul><li><b>AUTH_PAT</b></li><li><b>AUTH_OAUTH_M2M</b></li></ul>| [optional] [default to null] |
+| **auth\_options** | **List** | The available authentication methods that a partner can use to authenticate with Databricks. If it is not specified, `AUTH_PAT` will be used. The allowed options include <ul><li><b>AUTH_PAT</b></li><li><b>AUTH_OAUTH_M2M</b></li><li><b>AUTH_OAUTH_U2M</b></li></ul>| [optional] [default to null] |
 | **test\_workspace\_detail** | [**PartnerConfig_test_workspace_detail**](PartnerConfig_test_workspace_detail.md) |  | [optional] [default to null] |
+| **is_published_app** |  **Boolean**  | True if the partner app is registered as Databricks published OAuth app | [optional] [default to null] |
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
 

openapi/partner-connect-2.0.yaml

@@ -488,6 +488,12 @@ components:
             The secret of the service principal identity that a partner product can use to call Databricks APIs. 
             It will be set only when the auth_options in PartnerConfig contains the value AUTH_OAUTH_M2M.
           example: "secret"
+        oauth_u2m_app_id:
+          type: string
+          description: |
+            The client ID of OAuth U2M app connection (created by Partner Connect) that a partner product can use to initiate Databricks OAuth U2M flow. 
+            It will be set only when the auth_options in the PartnerConfig contains the value AUTH_OAUTH_U2M. 
+          example: "22c42f74-1dec-43d2-b649-3643f2e1e927"
         connection_scope:
           type: string
           description: The scope of users that can use this connection. Workspace means all users in the same workspace. User means only the user creating it.
@@ -543,6 +549,12 @@ components:
           type: boolean
           description: A boolean that represents whether the partner configured/persisted the Databricks resources on this Connect API request.  If the value is false and is_connection_established is false, Databricks will clean up the resources it provisioned
           example: true
+        oauth_redirect_uri:
+          type: string
+          description: |
+            The partner application's URL that handles Databricks OAuth redirect request in the OAuth U2M flow (Authorization code flow).
+            It should be set only when the partner is configured with OAuth U2M as auth option and does not have a pre-registered Databricks published OAuth app connection. 
+          example: https://www.partner.com/oauth/callback
     ErrorResponse:
       type: object
       required:

src/main/scala/com/databricks/partnerconnect/example/formatters/JsonFormatters.scala

@@ -67,7 +67,7 @@ object JsonFormatters extends DefaultJsonProtocol {
     new EnumJsonFormatter(TrialType)
   }
 
-  implicit val connectResponse: RootJsonFormat[Connection] = jsonFormat6(
+  implicit val connectResponse: RootJsonFormat[Connection] = jsonFormat7(
     Connection
   )
   implicit val auth: RootJsonFormat[Auth] = jsonFormat3(Auth)
@@ -152,7 +152,8 @@ object JsonFormatters extends DefaultJsonProtocol {
       ),
       "connection_scope" -> request.connection_scope
         .map(_.toJson)
-        .getOrElse(JsNull)
+        .getOrElse(JsNull),
+      "oauth_u2m_app_id" -> OptionJsString(request.oauth_u2m_app_id)
     )
 
     implicit val connectRequest: RootJsonFormat[ConnectRequest] =
@@ -193,7 +194,8 @@ object JsonFormatters extends DefaultJsonProtocol {
         service_principal_id = getOptionString(fields, "service_principal_id"),
         service_principal_oauth_secret =
           getOptionString(fields, "service_principal_oauth_secret"),
-        connection_scope = scoptOpt
+        connection_scope = scoptOpt,
+        oauth_u2m_app_id = getOptionString(fields, "oauth_u2m_app_id")
       )
     }
   }

src/test/scala/com/databricks/partnerconnect/client/tests/JsonFormattersTest.scala

@@ -42,11 +42,12 @@ class JsonFormattersTest extends PartnerTestBase {
     service_principal_id = Some("test-service-principal-id"),
     service_principal_oauth_secret =
       Some("test-service-principal-oauth-secret"),
-    connection_scope = Some(ConnectRequestEnums.ConnectionScope.Workspace)
+    connection_scope = Some(ConnectRequestEnums.ConnectionScope.Workspace),
+    oauth_u2m_app_id = Some("test-oauth-u2m-app-id")
   )
 
   val connectionRequestJson =
-    """{"catalog_name":"test-catalog-name","cloud_provider":"aws","cloud_provider_region":"test-cloud-provider-region","cluster_id":"test-cluster-id","connection_id":"test-connection-id","connection_scope":"workspace","data_source_connector":"test-data-source-connector","database_name":"test-database-name","databricks_jdbc_url":"jdbc://test-databricks-jdbc-url","demo":true,"destination_location":"test-destination-location","hostname":"test-hostname","http_path":"test-http-path","is_free_trial":true,"is_sql_endpoint":true,"is_sql_warehouse":true,"jdbc_url":"jdbc://test-jdcc-url","port":443,"service_principal_id":"test-service-principal-id","service_principal_oauth_secret":"test-service-principal-oauth-secret","user_info":{"databricks_organization_id":4645065419173783088,"databricks_user_id":5845867166711048519,"email":"[email protected]","first_name":"test-first-name","is_connection_established":false,"last_name":"test-last-name"},"workspace_id":1,"workspace_url":"https://test-workspace-url"}"""
+    """{"catalog_name":"test-catalog-name","cloud_provider":"aws","cloud_provider_region":"test-cloud-provider-region","cluster_id":"test-cluster-id","connection_id":"test-connection-id","connection_scope":"workspace","data_source_connector":"test-data-source-connector","database_name":"test-database-name","databricks_jdbc_url":"jdbc://test-databricks-jdbc-url","demo":true,"destination_location":"test-destination-location","hostname":"test-hostname","http_path":"test-http-path","is_free_trial":true,"is_sql_endpoint":true,"is_sql_warehouse":true,"jdbc_url":"jdbc://test-jdcc-url","oauth_u2m_app_id":"test-oauth-u2m-app-id","port":443,"service_principal_id":"test-service-principal-id","service_principal_oauth_secret":"test-service-principal-oauth-secret","user_info":{"databricks_organization_id":4645065419173783088,"databricks_user_id":5845867166711048519,"email":"[email protected]","first_name":"test-first-name","is_connection_established":false,"last_name":"test-last-name"},"workspace_id":1,"workspace_url":"https://test-workspace-url"}"""
 
   test(
     "serialize and deserialize ConnectRequest: All the fields are provided"