OAuth Authorization Code - Support client_id in request body as required by Azure AD

[elsewhat]

Testing restish client against an API using OAuth Authorization Code where the identity provider is Azure AD.
This authentication fails with the following message from Azure AD AADSTS900144: The request body must contain the following parameter: 'client_id'.

The following confguration is done for restish api configure <name>

? API auth type oauth-authorization-code
? Auth parameter client_id <client_id in Azure AD>
? Auth parameter authorize_url [? for help] https://login.windows.net/common/oauth2/authorize?resource=<resource id in Azure AD>
? Auth parameter token_url https://login.microsoftonline.com/common/oauth2/token

When executing a call, the browser is opened and Azure AD gives the error message AADSTS900144: The request body must contain the following parameter: 'client_id'.

The console output is

Open your browser to log in using the URL:
https://login.windows.net/common/oauth2/authorize?resource=<resource id>?response_type=code&code_challenge=<code_challenge>&code_challenge_method=S256&client_id=<client_id>&redirect_uri=http://localhost:8484/&scope=n

[danielgtaylor]

Interesting. The client_id is right there in the URL as a query param, but it seems to want it as part of the body. Maybe we need a way to specify whether to send it in the URL or body?

I'm open to suggestions here because I don't use Azure for auth. Most of my experience has been with Auth0.

[elsewhat]

@danielgtaylor have been troubleshooting a bit more and attempted to investigate how Microsoft expects the authorization to be done against Azure AD.

Get access on behalf of user from docs.microsoft.com describes the process well. And according to it, the initial request against the authorize_url should not have the contents in the body so the error message is a bit misleading.

However, realized there is an issue with the "Open your browser to log in" url created by restish against an Azure AD authorize_url. The authorize_url for Azure AD is of the format https://login.windows.net/common/oauth2/authorize?resource=<resource id in Azure AD> and therefore already has the ? character which starts the query string. Restish then adds a new ? character and the url outputted is non-conformant.
Instead of:

https://login.windows.net/common/oauth2/authorize?resource=<resource id>?response_type=code&code_challenge=<code_challenge>&code_challenge_method=S256&client_id=<client_id>&redirect_uri=http://localhost:8484/&scope=n

the url should be

https://login.windows.net/common/oauth2/authorize?resource=<resource id>&response_type=code&code_challenge=<code_challenge>&code_challenge_method=S256&client_id=<client_id>&redirect_uri=http://localhost:8484/&scope=n

Once this is fixed manually, the process proceeds further along.
However, for non-native apps, Azure AD requires a client_secret to be part of the POST body to the token_url. (a native app requires the reply url to be https://login.microsoftonline.com/common/oauth2/nativeclient which means it cannot be used with restish which needs the reply_url to be localhost)

Therefore, in order to support Azure AD authentication, there needs to be included a client_secret configuration parameter.
I see there is some code for client_secret in authcode.go, but when I attempt to configure the api from the command line as oauth-authorization-code it is never asked for. Could there be a bug when Required:false is used?
https://github.com/danielgtaylor/restish/blob/64590b908cb1abaea1eeb7c0da14d3e079796338/oauth/authcode.go#L162
The other optional parameter scopes is display (it has not Required:false property)
https://github.com/danielgtaylor/restish/blob/64590b908cb1abaea1eeb7c0da14d3e079796338/oauth/authcode.go#L165

The full output from restish when there is no client_secret is the following:

elsewhat@DESKTOP-4CF950R:~$ restish -v maintenance-api
DEBUG: Configuration: map[app-name:restish config-directory:/home/elsewhat/.restish rsh-filter: rsh-header:[] rsh-no-cache:false rsh-no-paginate:false rsh-output-format:auto rsh-profile:default rsh-query:[] rsh-raw:false rsh-server: rsh-verbose:false server-index:0]
DEBUG: Checking API entrypoint https://api-dev.gateway.equinor.com/maintenance-api/
Open your browser to log in using the URL:
https://login.windows.net/common/oauth2/authorize?resource=<resource id>?response_type=code&code_challenge=JLf7Zs_9o2TZGCs3ET867XoRIMOnKs9zGZ0-kPVwUVY&code_challenge_method=S256&client_id=<client_id>&redirect_uri=http://localhost:8484/&scope=
Alternatively, enter the code manually:
DEBUG: Making request:
POST /common/oauth2/token HTTP/1.1
Host: login.microsoftonline.com
Content-Type: application/x-www-form-urlencoded

grant_type=authorization_code&client_id=<client_id>&code_verifier=zu1byCmh-xlA966O_4Uwb33EntCMmRx40gTdP8nr9B0&code=0.AAAANaKkOuK21UiRlX_PBbRZsAFtZQe90WFIsWnikCVD6ZcCAP4.AQABAAIAAABeStGSRwwnTq2vHplZ9KL4__m94RBqMcAh_BsU-Sg53HZpra5ND9gATJWk32tynGgO4N_Amt9LuPkXnpCluyxq9-0rUZJVAUlBHJsfFH3e9D3OBFmsE3-AHT6agJ8akS6XtxdoeWgajTjdy6VkO3yQVzu-hjcY....&redirect_uri=http://localhost:8484/
DEBUG: Got response from server in 380.0296ms:
HTTP/1.1 401 Unauthorized
Content-Length: 650
Cache-Control: no-store, no-cache
Content-Type: application/json; charset=utf-8
Date: Tue, 05 Jan 2021 12:34:36 GMT
Expires: -1
P3p: CP="DSP CUR OTPi IND OTRi ONL FIN"
Pragma: no-cache
Set-Cookie: fpc=...; expires=Thu, 04-Feb-2021 12:34:36 GMT; path=/; secure; HttpOnly; SameSite=None
Set-Cookie: x-ms-gateway-slice=prod; path=/; secure; samesite=none; httponly
Set-Cookie: stsservicecookie=ests; path=/; secure; samesite=none; httponly
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Ms-Ests-Server: 2.1.11328.14 - AMS1 ProdSlices
X-Ms-Request-Id: 97310b6e-a7ae-4f08-be34-e91f9f1f2300

{"error":"invalid_client","error_description":"AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.\r\nTrace ID: 97310b6e-a7ae-4f08-be34-e91f9f1f2300\r\nCorrelation ID: cd969d60-32b4-4ddf-bf6d-1b826c7d5456\r\nTimestamp: 2021-01-05 12:34:36Z","error_codes":[7000218],"timestamp":"2021-01-05 12:34:36Z","trace_id":"97310b6e-a7ae-4f08-be34-e91f9f1f2300","correlation_id":"cd969d60-32b4-4ddf-bf6d-1b826c7d5456","error_uri":"https://login.microsoftonline.com/error?code=7000218","claims":"{\"access_token\":{\"capolids\":{\"essential\":true,\"values\":[\"6175d910-f18b-421a-b85c-950e1841952c\"]}}}"}
panic: bad response from token endpoint:
{"error":"invalid_client","error_description":"AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.\r\nTrace ID: 97310b6e-a7ae-4f08-be34-e91f9f1f2300\r\nCorrelation ID: cd969d60-32b4-4ddf-bf6d-1b826c7d5456\r\nTimestamp: 2021-01-05 12:34:36Z","error_codes":[7000218],"timestamp":"2021-01-05 12:34:36Z","trace_id":"97310b6e-a7ae-4f08-be34-e91f9f1f2300","correlation_id":"cd969d60-32b4-4ddf-bf6d-1b826c7d5456","error_uri":"https://login.microsoftonline.com/error?code=7000218","claims":"{\"access_token\":{\"capolids\":{\"essential\":true,\"values\":[\"6175d910-f18b-421a-b85c-950e1841952c\"]}}}"}

goroutine 1 [running]:
github.com/danielgtaylor/restish/cli.MakeRequest(0xc00050c400, 0xc000191a68, 0x2, 0x2, 0x0, 0x0, 0x0)
        /home/runner/work/restish/restish/cli/request.go:91 +0x1508
github.com/danielgtaylor/restish/cli.Load(0xc00002ab00, 0x33, 0xc000223b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/runner/work/restish/restish/cli/api.go:143 +0x490
github.com/danielgtaylor/restish/cli.Run()
        /home/runner/work/restish/restish/cli/cli.go:446 +0x2d7
main.main()
        /home/runner/work/restish/restish/main.go:27 +0x177

To summarize, there are two fixes to be made in order to support Azure AD.

1. If the authorize_url already contains a query string, output & instead of ?
   https://github.com/danielgtaylor/restish/blob/64590b908cb1abaea1eeb7c0da14d3e079796338/oauth/authcode.go#L98

2. Allow client_secret to be configured and include it in POST payload to token_url

[danielgtaylor]

@elsewhat please have a look at e1d9b7d which has more robust URI and POST payload generation code added. I believe the latest release should fix both problems, I am asked for an optional client secret when using the interactive prompts.

[elsewhat]

Awesome @danielgtaylor !

Tested version 0.6.0 and it's very close to working. However, it fails when parsing the successful authentication from Azure AD.

The error is
panic: json: cannot unmarshal string into Go struct field tokenResponse.expires_in of type time.Duration

Microsoft documents the token response here: https://docs.microsoft.com/en-us/graph/auth-v2-user#token-response
"expires_in - How long the access token is valid (in seconds)"

Here is the relevant output when running restish -v

HTTP/1.1 200 OK
Content-Length: 3907
Cache-Control: no-store, no-cache
Content-Type: application/json; charset=utf-8
Date: Tue, 02 Feb 2021 16:18:21 GMT
Expires: -1
P3p: CP="DSP CUR OTPi IND OTRi ONL FIN"
Pragma: no-cache
Set-Cookie: fpc=<some cookie value>; expires=Thu, 04-Mar-2021 16:18:22 GMT; path=/; secure; HttpOnly; SameSite=None
Set-Cookie: x-ms-gateway-slice=prod; path=/; secure; samesite=none; httponly
Set-Cookie: stsservicecookie=ests; path=/; secure; samesite=none; httponly
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Ms-Ests-Server: 2.1.11444.12 - AMS1 ProdSlices
X-Ms-Request-Id: cf0cebb3-9b71-4f89-8d03-0c8fdba11b00

{"token_type":"Bearer","scope":"MaintenanceAPITest","expires_in":"3599","ext_expires_in":"3599","expires_on":"1612286302","not_before":"1612282402","resource":"<resource id>","access_token":"<access token value>","refresh_token":"<refresh token value>"}
panic: json: cannot unmarshal string into Go struct field tokenResponse.expires_in of type time.Duration

goroutine 1 [running]:
github.com/danielgtaylor/restish/cli.MakeRequest(0xc00069ab00, 0xc00034d9a0, 0x2, 0x2, 0x0, 0x0, 0x0)
        /home/runner/work/restish/restish/cli/request.go:91 +0x1508
github.com/danielgtaylor/restish/cli.Load(0xc00002af40, 0x33, 0xc0001d5b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/runner/work/restish/restish/cli/api.go:144 +0x4a8
github.com/danielgtaylor/restish/cli.Run()
        /home/runner/work/restish/restish/cli/cli.go:459 +0x2d0
main.main()
        /home/runner/work/restish/restish/main.go:27 +0x177