Support authorised access to Data Sources

[tronboto]

Is your feature request related to a problem? Please describe.

The Data Sources feature is great, however, it currently doesn't seem to be possible to control an individual user's access to data in remote data sources.

Describe the solution you'd like

It would be great if access to remote Opensearch data sources was granted based on the principal of the logged in user and any roles they have mapped in the remote cluster.

Describe alternatives you've considered

Using the current solution, which uses the roles that are mapped to the user configured on the data source. However, we need to be able to restrict a user's access to be the same as if they were actually logged into the remote cluster.

Additional context

N/A

[ananzh]

@bandinib-amzn could you help to take a look?

[seraphjiang]

In general, data access policy should be managed by the remote database themselves.

Who(principal) could have what permission/access to what data is not part of datasource and won't be part of datasource.

We do respect existing access policy. Hope this helped clarification

Share identity and principal only happens when both OSD and datasource are using same identity Provider. Most of time is might not feasible e.g. OSD might use OIDC for sso, backend might use user/password

[seraphjiang]

@bandinib-amzn could you help to take a look?
@ananzh feel free to tag me in the future for MDS related bug/feature request.

[tronboto]

Hey @seraphjiang

We do respect existing access policy. Hope this helped clarification

Maybe I'm misunderstanding but this doesn't seem to be working for me on Opensearch 2.11.1 then. On the cluster with data_source.enabled: true, I am able to run a command, for example, GET _cat/indices in Dev Tools against the remote cluster, however, when I try to do the same from Dev Tools in the remote cluster itself, I get:

no permissions for [indices:monitor/settings/get] and User...

which is correct - the user I'm logged in as on the remote cluster doesn't have permission to do that. Note that I am logged into both Opensearch Dashboards instances with the same principal using OIDC.

I think you can also see this behaviour at: https://playground.opensearch.org/app/dev_tools#/console

If you go there and run GET _cat/indices, you get the error:

no permissions for [indices:monitor/settings/get] and User [name=mdPlayGround, backend_roles=[], requestedTenant=null]"

where mdPlayGround is presumably the user configured on the data source. At least, it's not the logged in user, which is opendistro_security_anonymous.

[kuzaxak]

I can confirm that Opensearch Dashboard do not forward any auth credentials if SSO is used. I would expect that it will proxy user token to downstream datasource. This way downstream datasource will be able to authorise a request.

As far as I understand right now only static auth is supported and proxy auth isn't supported yet. I'm right?

[seraphjiang]

In order to support pass through credential from upstream to downstream securely. OSD need to encrypt the credential and downstream need to be able to decrypt the credential.

We could see if there is more needs for this feature request

Cc: @zengyan-amazon @elfisher