Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3b75a09
Init Oauth token authentication
devnied Oct 6, 2024
bce57a2
Optimize images
devnied Oct 6, 2024
8a477f9
Implement OAuth flow
devnied Oct 7, 2024
7ce34b9
Update go.mod and go.sum
devnied Oct 7, 2024
91fa5ec
Init Oauth token authentication
devnied Oct 6, 2024
c4c3d31
Optimize images
devnied Oct 6, 2024
4cfa0c4
Implement OAuth flow
devnied Oct 7, 2024
25908c5
Add clientId & clientSecret
devnied Oct 14, 2024
3e0bd84
dep: Add oauth2 dependencies
devnied Oct 17, 2024
be17ada
Merge remote-tracking branch 'origin/master' into feature/rework-auth…
devnied Nov 4, 2024
48cc4fd
Merge branch 'feature/rework-authentication' of https://github.com/mi…
devnied Nov 4, 2024
cb2d7db
Merge branch 'master' into feature/rework-authentication
devnied Nov 7, 2024
4142116
Merge branch 'master' into feature/rework-authentication
devnied Nov 13, 2024
f3fdb8f
chore: merge from master branch
devnied Jan 14, 2025
1c53cf6
chore: upgrade go dependencies
devnied Jan 15, 2025
1635309
feature: add oAuth flow to connection validation
devnied Jan 21, 2025
f192c34
chore: update documentation
devnied Jan 21, 2025
71b9821
feat: Rework authentication secret parameters
devnied Jan 21, 2025
f447021
chore: rework package and add unit tests
devnied Jan 21, 2025
c9ddd93
chore: Add unit tests for Oauth methods
devnied Jan 21, 2025
cb67544
chore: Fix sonar violations
devnied Jan 22, 2025
297b413
Use AppUrl constant
devnied Feb 10, 2025
8117397
Add support of client credential Oauth flow
devnied Feb 12, 2025
4f7b1c3
Update dependencies
devnied Feb 12, 2025
21d48c0
Merge branch 'master' into feature/rework-authentication
devnied Feb 12, 2025
b272610
refactor: streamline health check error handling and validation logic
devnied Feb 12, 2025
bdd4954
refactor: remove unreachable code
devnied Feb 13, 2025
f4ff9f4
Update changelog
devnied Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v5
with:
go-version: "1.22"
go-version: "1.23"

- name: Test backend
uses: magefile/mage-action@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v5
with:
go-version: "1.22"
go-version: "1.23"

- name: Get yarn cache directory path
id: yarn-cache-dir-path
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ e2e-results/

.DS_Store
.bra.toml

.eslintcache
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,19 @@

### ⭐ Added
- Add a query tag that includes relevant Grafana context information.
- Add support of OAuth authentication.

### 🐞 Bug Fixes
- Source query variables are replaced with hardcoded values in the query editor UI.

### 🔨 Changed
- Rewrite the datasource configuration UI (ease authentication selection).
- Support non encoded private key in the datasource configuration.
- Update deprecated APIs
- Upgrade grafana-plugin-sdk-go to version v0.265.0.
- Upgrade gosnowflake to version v1.13.0.
- Upgrade go to version 1.23.

## 1.9.1

### 🔨 Changed
Expand Down
67 changes: 56 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,15 @@ docker run -d \
grafana/grafana
```

> [!NOTE]
> Please refer to the documentation for more details.
https://grafana.com/docs/grafana/latest/administration/plugin-management/#allow-unsigned-plugins

3. Restart grafana
Restart the Grafana server to apply the changes:
``` bash
service grafana-server restart
```

#### Configure the Datasource

Expand All @@ -57,17 +65,43 @@ Add your authentication and [configuration details](https://docs.snowflake.com/e

Available configuration fields are as follows:

Name | Description
------------------------- | ------------
Account Name | Specifies the full name of your account (provided by Snowflake)
Username | Specifies the login name of the user for the connection.
Password | Specifies the password for the specified user.
Private key | Specifies the the private key. Must be encoded in base 64 URL encoded pkcs8.<br/>**Command :**<br/> `egrep -v '^(-----BEGIN PRIVATE KEY\|-----END PRIVATE KEY)' rsa_key.p8 \| tr -d '\n' \| sed 's/+/-/g; s/\//_/g' > rsa_key_urlbase64.p8`
Role (Optional) | Specifies the default access control role to use in the Snowflake session initiated by Grafana.
Warehouse (Optional) | Specifies the virtual warehouse to use once connected.
Database (Optional) | Specifies the default database to use once connected.
Schema (Optional) | Specifies the default schema to use for the specified database once connected.
Extra Options (Optional) | Specifies a series of one or more parameters, in the form of `<param>=<value>`, with each parameter separated by the ampersand character (&), and no spaces anywhere in the connection string.
| Name | Description |
|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Account Name | Specifies the full name of your account (provided by Snowflake) |
| Username | Specifies the login name of the user for the connection. |
| Password | Specifies the password for the specified user. |
| Private key | Specifies the private key. |
| Client Id | Specifies the Oauth client ID. |
| Client Secret | Specifies the Oauth client Secret. |
| Token Endpoint | Specifies the Oauth Token endpoint. |
| Role (Optional) | Specifies the default access control role to use in the Snowflake session initiated by Grafana. With Oauth, it's used to limit the access token to a single role that the user can consent to for the session. |
| Warehouse (Optional) | Specifies the virtual warehouse to use once connected. |
| Database (Optional) | Specifies the default database to use once connected. |
| Schema (Optional) | Specifies the default schema to use for the specified database once connected. |
| Extra Options (Optional) | Specifies a series of one or more parameters, in the form of `<param>=<value>`, with each parameter separated by the ampersand character (&), and no spaces anywhere in the connection string. |

**External OAuth authentication**

> [!NOTE]
> Snowflake oauth authentication is not supported without external service (like Okta, Azure Entra, Keycloak ...) because of the lack of support for oauth Client credentials flow in snowflake.
https://docs.snowflake.com/en/user-guide/oauth-intro

The plugin supports OAuth authentication with snowflake only with external_service.<br/>
To use OAuth, you need to create an [external OAuth](https://docs.snowflake.com/en/user-guide/oauth-ext-custom) integration in your Snowflake account.
```sql
-- Create a security integration for external OAuth flow
CREATE OR REPLACE SECURITY INTEGRATION OAUTH_INTEGRATION
TYPE = EXTERNAL_OAUTH
ENABLED = TRUE
EXTERNAL_OAUTH_TYPE = CUSTOM
EXTERNAL_OAUTH_SCOPE_MAPPING_ATTRIBUTE = 'scope'
EXTERNAL_OAUTH_TOKEN_USER_MAPPING_CLAIM = 'name'
EXTERNAL_OAUTH_SNOWFLAKE_USER_MAPPING_ATTRIBUTE = 'login_name'
EXTERNAL_OAUTH_ALLOWED_ROLES_LIST = ('<xxxxx>')
EXTERNAL_OAUTH_AUDIENCE_LIST =('https://xxxxx')
EXTERNAL_OAUTH_RSA_PUBLIC_KEY = 'MIIBIj'
EXTERNAL_OAUTH_ISSUER = 'https://xxxxx';
```

#### Supported Macros

Expand Down Expand Up @@ -108,6 +142,9 @@ For Time series query:

![Query editor](img/query.png)

> [!CAUTION]
> This plugin cannot identify malicious code in queries executed on Snowflake and assumes no responsibility for their execution. As a precaution, use a ROLE with minimal privileges, configured to grant read-only access

##### Query Variables

You can use query variable in your Snowflake queries by using [variable syntax](https://grafana.com/docs/grafana/latest/dashboards/variables/variable-syntax/).<br/>
Expand Down Expand Up @@ -187,13 +224,21 @@ GROUP BY
Annotations allow you to overlay events on a graph.
To create an annotation, in the dashboard settings click "Annotations", and "New".

#### Oauth Configuration

To use Oauth, you need to create an Oauth custom integration in your Snowflake account.<
You can follow the steps in the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/oauth-custom).

## Caching
### Snowflake caching

Snowflake caches queries with the same footprint / hash in its own query-cache. Since a Grafana query mostly has a now() component the cache will never be used.
To get more queries with the same hash use the two macros `$__timeRoundFrom(d)` and `$__timeRoundTo(d)` to create wider truncated timestamps. This is no problem for timeseries charts. Grafana cuts it's x-Axis to the selected dashboard time window. If a table is displayed the whole result will be presented and it could be slightly out of the time window.\
More info about snowflake-side caching: https://docs.snowflake.com/en/user-guide/querying-persisted-results#retrieval-optimization

## Supported Grafana Versions
This plugin supports only version with [Active Support from Grafana](https://grafana.com/docs/grafana/next/upgrade-guide/when-to-upgrade/?pg=blog&plcmt=body-txt#what-to-know-about-version-support).

## Development

The snowflake datasource is a data source backend plugin composed of both frontend and backend components.
Expand Down
81 changes: 42 additions & 39 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
module github.com/michelin/snowflake-grafana-datasource

go 1.22
go 1.23.5

toolchain go1.22.0
toolchain go1.23.6

require (
github.com/DATA-DOG/go-sqlmock v1.5.2
github.com/grafana/grafana-plugin-sdk-go v0.260.1
github.com/grafana/grafana-plugin-sdk-go v0.265.0
github.com/snowflakedb/gosnowflake v1.13.0
github.com/stretchr/testify v1.10.0
golang.org/x/oauth2 v0.26.0
)

require (
Expand All @@ -19,7 +20,7 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0 // indirect
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c // indirect
github.com/apache/arrow/go/v15 v15.0.2 // indirect
github.com/apache/arrow-go/v18 v18.0.1-0.20241212180703-82be143d7c30 // indirect
github.com/apache/arrow/go/v16 v16.0.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.26.1 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.2 // indirect
Expand All @@ -43,38 +44,37 @@ require (
github.com/danieljoos/wincred v1.1.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dvsekhvalnov/jose2go v1.6.0 // indirect
github.com/elazarl/goproxy v0.0.0-20230731152917-f99041a5c027 // indirect
github.com/elazarl/goproxy v1.7.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.7 // indirect
github.com/getkin/kin-openapi v0.128.0 // indirect
github.com/getkin/kin-openapi v0.129.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/goccy/go-json v0.10.4 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/flatbuffers v24.3.25+incompatible // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/grafana/otel-profiling-go v0.5.1 // indirect
github.com/grafana/pyroscope-go/godeltaprof v0.1.8 // indirect
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1 // indirect
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.23.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.25.1 // indirect
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-plugin v1.6.2 // indirect
github.com/hashicorp/go-plugin v1.6.3 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/invopop/yaml v0.3.1 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/magefile/mage v1.15.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattetti/filebuffer v1.0.1 // indirect
Expand All @@ -86,15 +86,17 @@ require (
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/mtibben/percent v0.2.1 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/oasdiff/yaml v0.0.0-20241210131133-6b86fb107d80 // indirect
github.com/oasdiff/yaml3 v0.0.0-20241210130736-a94c01f36349 // indirect
github.com/oklog/run v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/perimeterx/marshmallow v1.1.5 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.20.5 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.60.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
Expand All @@ -103,31 +105,32 @@ require (
github.com/unknwon/log v0.0.0-20150304194804-e617c87089d3 // indirect
github.com/urfave/cli v1.22.16 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.56.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.57.0 // indirect
go.opentelemetry.io/contrib/propagators/jaeger v1.32.0 // indirect
go.opentelemetry.io/contrib/samplers/jaegerremote v0.26.0 // indirect
go.opentelemetry.io/otel v1.32.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.32.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.32.0 // indirect
go.opentelemetry.io/otel/metric v1.32.0 // indirect
go.opentelemetry.io/otel/sdk v1.32.0 // indirect
go.opentelemetry.io/otel/trace v1.32.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
golang.org/x/crypto v0.31.0 // indirect
golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/term v0.27.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.59.0 // indirect
go.opentelemetry.io/contrib/propagators/jaeger v1.34.0 // indirect
go.opentelemetry.io/contrib/samplers/jaegerremote v0.28.0 // indirect
go.opentelemetry.io/otel v1.34.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0 // indirect
go.opentelemetry.io/otel/metric v1.34.0 // indirect
go.opentelemetry.io/otel/sdk v1.34.0 // indirect
go.opentelemetry.io/otel/trace v1.34.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/exp v0.0.0-20240909161429-701f63a606c0 // indirect
golang.org/x/mod v0.22.0 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sync v0.11.0 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/term v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
golang.org/x/tools v0.28.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20241104194629-dd2ea8efbc28 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241104194629-dd2ea8efbc28 // indirect
google.golang.org/grpc v1.67.1 // indirect
google.golang.org/protobuf v1.35.2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250115164207-1a7da9e5054f // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f // indirect
google.golang.org/grpc v1.70.0 // indirect
google.golang.org/protobuf v1.36.4 // indirect
gopkg.in/fsnotify/fsnotify.v1 v1.4.7 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading