Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/BIGQUERY_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ The BigQuery MCP server is configured using environment variables.
```bash
export BIGQUERY_PROJECT="<your-gcp-project-id>"
export BIGQUERY_LOCATION="<your-dataset-location>" # Optional
export BIGQUERY_USE_CLIENT_OAUTH="true" # Optional
export BIGQUERY_USE_CLIENT_OAUTH="true" # Optional: true, false, or a custom header name
export BIGQUERY_SCOPES="<comma-separated-scopes>" # Optional
```

Expand Down
15 changes: 10 additions & 5 deletions docs/en/resources/sources/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,10 +104,15 @@ Common scopes include `https://www.googleapis.com/auth/bigquery` or
### Authentication via User's OAuth Access Token

If the `useClientOAuth` parameter is set to `true`, Toolbox will instead use the
OAuth access token for authentication. This token is parsed from the
`Authorization` header passed in with the tool invocation request. This method
allows Toolbox to make queries to [BigQuery][bigquery-docs] on behalf of the
client or the end-user.
OAuth access token for authentication. By default, this token is parsed from the
`Authorization` header passed in with the tool invocation request.

If you need to use a non-standard header for the access token (e.g., to avoid
conflicts with other services like Cloud Run), you can specify the header name
in the `useClientOAuth` field (e.g., `useClientOAuth: X-BigQuery-Auth`).

This method allows Toolbox to make queries to [BigQuery][bigquery-docs] on behalf
of the client or the end-user.

When using this on-behalf-of authentication, you must ensure that the
identity used has been granted the correct IAM permissions.
Expand Down Expand Up @@ -166,7 +171,7 @@ useClientOAuth: true
| location | string | false | Specifies the location (e.g., 'us', 'asia-northeast1') in which to run the query job. This location must match the location of any tables referenced in the query. Defaults to the table's location or 'US' if the location cannot be determined. [Learn More](https://cloud.google.com/bigquery/docs/locations) |
| writeMode | string | false | Controls the write behavior for tools. `allowed` (default): All queries are permitted. `blocked`: Only `SELECT` statements are allowed for the `bigquery-execute-sql` tool. `protected`: Enables session-based execution where all tools associated with this source instance share the same [BigQuery session](https://cloud.google.com/bigquery/docs/sessions-intro). This allows for stateful operations using temporary tables (e.g., `CREATE TEMP TABLE`). For `bigquery-execute-sql`, `SELECT` statements can be used on all tables, but write operations are restricted to the session's temporary dataset. For tools like `bigquery-sql`, `bigquery-forecast`, and `bigquery-analyze-contribution`, the `writeMode` restrictions do not apply, but they will operate within the shared session. **Note:** The `protected` mode cannot be used with `useClientOAuth: true`. It is also not recommended for multi-user server environments, as all users would share the same session. A session is terminated automatically after 24 hours of inactivity or after 7 days, whichever comes first. A new session is created on the next request, and any temporary data from the previous session will be lost. |
| allowedDatasets | []string | false | An optional list of dataset IDs that tools using this source are allowed to access. If provided, any tool operation attempting to access a dataset not in this list will be rejected. To enforce this, two types of operations are also disallowed: 1) Dataset-level operations (e.g., `CREATE SCHEMA`), and 2) operations where table access cannot be statically analyzed (e.g., `EXECUTE IMMEDIATE`, `CREATE PROCEDURE`). If a single dataset is provided, it will be treated as the default for prebuilt tools. |
| useClientOAuth | bool | false | If true, forwards the client's OAuth access token from the "Authorization" header to downstream queries. **Note:** This cannot be used with `writeMode: protected`. |
| useClientOAuth | string | false | If set to `'true'`, forwards the client's OAuth access token from the default `Authorization` header. If set to a custom header name (e.g., `X-My-Auth`), that header will be used instead. An empty string or `'false'` disables this feature. Defaults to `""` (disabled). |
Comment thread
shobsi marked this conversation as resolved.
| scopes | []string | false | A list of OAuth 2.0 scopes to use for the credentials. If not provided, default scopes are used. |
| impersonateServiceAccount | string | false | Service account email to impersonate when making BigQuery and Dataplex API calls. The authenticated principal must have the `roles/iam.serviceAccountTokenCreator` role on the target service account. [Learn More](https://cloud.google.com/iam/docs/service-account-impersonation) |
| maxQueryResultRows | int | false | The maximum number of rows to return from a query. Defaults to 50. |
2 changes: 1 addition & 1 deletion docs/en/resources/sources/looker.md
Comment thread
shobsi marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ instead of hardcoding your secrets into the configuration file.
| project | string | false | The project id to use in Google Cloud. |
| location | string | false | The location to use in Google Cloud. (default: us) |
| timeout | string | false | Maximum time to wait for query execution (e.g. "30s", "2m"). By default, 120s is applied. |
| use_client_oauth | string | false | Use OAuth tokens instead of client_id and client_secret. (default: false) If a header name is provided, it will be used instead of "Authorization". |
| use_client_oauth | string | false | If set to `'true'`, forwards the client's OAuth access token from the default `Authorization` header. If set to a custom header name (e.g., `X-Looker-Auth`), that header will be used instead. An empty string or `'false'` disables this feature. Defaults to `""` (disabled). |
| show_hidden_models | string | false | Show or hide hidden models. (default: true) |
| show_hidden_explores | string | false | Show or hide hidden explores. (default: true) |
| show_hidden_fields | string | false | Show or hide hidden fields. (default: true) |
58 changes: 36 additions & 22 deletions internal/sources/bigquery/bigquery.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ type Config struct {
Location string `yaml:"location"`
WriteMode string `yaml:"writeMode"`
AllowedDatasets StringOrStringSlice `yaml:"allowedDatasets"`
UseClientOAuth bool `yaml:"useClientOAuth"`
UseClientOAuth string `yaml:"useClientOAuth"`
ImpersonateServiceAccount string `yaml:"impersonateServiceAccount"`
Scopes StringOrStringSlice `yaml:"scopes"`
MaxQueryResultRows int `yaml:"maxQueryResultRows"`
Expand Down Expand Up @@ -123,6 +123,7 @@ func (r Config) SourceConfigType() string {
// Returns BigQuery source type
return SourceType
}

func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error) {
if r.WriteMode == "" {
r.WriteMode = WriteModeAllowed
Expand All @@ -132,15 +133,15 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
r.MaxQueryResultRows = 50
}

if r.WriteMode == WriteModeProtected && r.UseClientOAuth {
if r.WriteMode == WriteModeProtected && strings.ToLower(r.UseClientOAuth) != "false" && r.UseClientOAuth != "" {
// The protected mode only allows write operations to the session's temporary datasets.
// when using client OAuth, a new session is created every
// time a BigQuery tool is invoked. Therefore, no session data can
// be preserved as needed by the protected mode.
return nil, fmt.Errorf("writeMode 'protected' cannot be used with useClientOAuth 'true'")
return nil, fmt.Errorf("writeMode 'protected' cannot be used with useClientOAuth enabled")
}

if r.UseClientOAuth && r.ImpersonateServiceAccount != "" {
if strings.ToLower(r.UseClientOAuth) != "false" && r.UseClientOAuth != "" && r.ImpersonateServiceAccount != "" {
return nil, fmt.Errorf("useClientOAuth cannot be used with impersonateServiceAccount")
}

Expand All @@ -151,23 +152,16 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
var err error

s := &Source{
Config: r,
Client: client,
RestService: restService,
TokenSource: tokenSource,
MaxQueryResultRows: r.MaxQueryResultRows,
ClientCreator: clientCreator,
Config: r,
Client: client,
RestService: restService,
TokenSource: tokenSource,
MaxQueryResultRows: r.MaxQueryResultRows,
ClientCreator: clientCreator,
AuthTokenHeaderName: "Authorization",
}

if r.UseClientOAuth {
// use client OAuth
baseClientCreator, err := newBigQueryClientCreator(ctx, tracer, r.Project, r.Location, r.Name)
if err != nil {
return nil, fmt.Errorf("error constructing client creator: %w", err)
}
setupClientCaching(s, baseClientCreator)

} else {
if strings.ToLower(r.UseClientOAuth) == "false" || r.UseClientOAuth == "" {
// Initializes a BigQuery Google SQL source
client, restService, tokenSource, err = initBigQueryConnection(ctx, tracer, r.Name, r.Project, r.Location, r.ImpersonateServiceAccount, r.Scopes)
if err != nil {
Expand All @@ -176,6 +170,21 @@ func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.So
s.Client = client
s.RestService = restService
s.TokenSource = tokenSource

if r.WriteMode == WriteModeProtected {
// session-based connections
s.SessionProvider = s.newBigQuerySessionProvider()
}
} else {
if strings.ToLower(r.UseClientOAuth) != "true" {
s.AuthTokenHeaderName = r.UseClientOAuth
}
// use client OAuth
baseClientCreator, err := newBigQueryClientCreator(ctx, tracer, r.Project, r.Location, r.Name)
if err != nil {
return nil, fmt.Errorf("error constructing client creator: %w", err)
}
setupClientCaching(s, baseClientCreator)
}

allowedDatasets := make(map[string]struct{})
Expand Down Expand Up @@ -280,6 +289,7 @@ type Source struct {
Client *bigqueryapi.Client
RestService *bigqueryrestapi.Service
TokenSource oauth2.TokenSource
AuthTokenHeaderName string
MaxQueryResultRows int
ClientCreator BigqueryClientCreator
AllowedDatasets map[string]struct{}
Expand Down Expand Up @@ -417,7 +427,11 @@ func (s *Source) newBigQuerySessionProvider() BigQuerySessionProvider {
}

func (s *Source) UseClientAuthorization() bool {
return s.UseClientOAuth
return strings.ToLower(s.UseClientOAuth) != "false" && s.UseClientOAuth != ""
}

func (s *Source) GetAuthTokenHeaderName() string {
return s.AuthTokenHeaderName
}

func (s *Source) BigQueryProject() string {
Expand Down Expand Up @@ -497,15 +511,15 @@ func (s *Source) lazyInitDataplexClient(ctx context.Context, tracer trace.Tracer

return func() (*dataplexapi.CatalogClient, DataplexClientCreator, error) {
once.Do(func() {
c, cc, e := initDataplexConnection(ctx, tracer, s.Name, s.Project, s.UseClientOAuth, s.ImpersonateServiceAccount, s.Scopes)
c, cc, e := initDataplexConnection(ctx, tracer, s.Name, s.Project, s.UseClientAuthorization(), s.ImpersonateServiceAccount, s.Scopes)
if e != nil {
err = fmt.Errorf("failed to initialize dataplex client: %w", e)
return
}
client = c

// If using OAuth, wrap the provided client creator (cc) with caching logic
if s.UseClientOAuth && cc != nil {
if s.UseClientAuthorization() && cc != nil {
clientCreator = func(tokenString string) (*dataplexapi.CatalogClient, error) {
// Check cache
if val, found := s.dataplexCache.Get(tokenString); found {
Expand Down
68 changes: 64 additions & 4 deletions internal/sources/bigquery/bigquery_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ func TestParseFromYamlBigQuery(t *testing.T) {
Project: "my-project",
Location: "asia",
WriteMode: "blocked",
UseClientOAuth: false,
UseClientOAuth: "",
},
},
},
Expand All @@ -82,6 +82,46 @@ func TestParseFromYamlBigQuery(t *testing.T) {
type: bigquery
project: my-project
location: us
useClientOAuth: "true"
`,
want: map[string]sources.SourceConfig{
"my-instance": bigquery.Config{
Name: "my-instance",
Type: bigquery.SourceType,
Project: "my-project",
Location: "us",
UseClientOAuth: "true",
},
},
},
{
desc: "with custom auth header name example",
in: `
kind: sources
name: my-instance
type: bigquery
project: my-project
location: us
useClientOAuth: X-Custom-Auth
`,
want: map[string]sources.SourceConfig{
"my-instance": bigquery.Config{
Name: "my-instance",
Type: bigquery.SourceType,
Project: "my-project",
Location: "us",
UseClientOAuth: "X-Custom-Auth",
},
},
},
{
desc: "use client auth with unquoted true",
in: `
kind: sources
name: my-instance
type: bigquery
project: my-project
location: us
useClientOAuth: true
`,
want: map[string]sources.SourceConfig{
Expand All @@ -90,7 +130,27 @@ func TestParseFromYamlBigQuery(t *testing.T) {
Type: bigquery.SourceType,
Project: "my-project",
Location: "us",
UseClientOAuth: true,
UseClientOAuth: "true",
},
},
},
{
desc: "use client auth with unquoted false",
in: `
kind: sources
name: my-instance
type: bigquery
project: my-project
location: us
useClientOAuth: false
`,
want: map[string]sources.SourceConfig{
"my-instance": bigquery.Config{
Name: "my-instance",
Type: bigquery.SourceType,
Project: "my-project",
Location: "us",
UseClientOAuth: "false",
},
},
},
Expand Down Expand Up @@ -253,7 +313,7 @@ func TestInitialize_MaxQueryResultRows(t *testing.T) {
Name: "test-default",
Type: bigquery.SourceType,
Project: "test-project",
UseClientOAuth: true,
UseClientOAuth: "true",
},
want: 50,
},
Expand All @@ -263,7 +323,7 @@ func TestInitialize_MaxQueryResultRows(t *testing.T) {
Name: "test-configured",
Type: bigquery.SourceType,
Project: "test-project",
UseClientOAuth: true,
UseClientOAuth: "true",
MaxQueryResultRows: 100,
},
want: 100,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.T
type compatibleSource interface {
BigQueryClient() *bigqueryapi.Client
UseClientAuthorization() bool
GetAuthTokenHeaderName() string
IsDatasetAllowed(projectID, datasetID string) bool
BigQueryAllowedDatasets() []string
BigQuerySession() bigqueryds.BigQuerySessionProvider
Expand Down Expand Up @@ -340,7 +341,11 @@ func (t Tool) RequiresClientAuthorization(resourceMgr tools.SourceProvider) (boo
}

func (t Tool) GetAuthTokenHeaderName(resourceMgr tools.SourceProvider) (string, error) {
return "Authorization", nil
source, err := tools.GetCompatibleSource[compatibleSource](resourceMgr, t.Source, t.Name, t.Type)
if err != nil {
return "", err
}
return source.GetAuthTokenHeaderName(), nil
}

func (t Tool) GetParameters() parameters.Parameters {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ type compatibleSource interface {
BigQueryLocation() string
GetMaxQueryResultRows() int
UseClientAuthorization() bool
GetAuthTokenHeaderName() string
IsDatasetAllowed(projectID, datasetID string) bool
BigQueryAllowedDatasets() []string
}
Expand Down Expand Up @@ -241,9 +242,9 @@ func (t Tool) Invoke(ctx context.Context, resourceMgr tools.SourceProvider, para
caURL := fmt.Sprintf(gdaURLFormat, projectID, location)

headers := map[string]string{
"Authorization": fmt.Sprintf("Bearer %s", tokenStr),
"Content-Type": "application/json",
"X-Goog-API-Client": util.GDAClientID,
source.GetAuthTokenHeaderName(): fmt.Sprintf("Bearer %s", tokenStr),
"Content-Type": "application/json",
"X-Goog-API-Client": util.GDAClientID,
}

payload := CAPayload{
Expand Down Expand Up @@ -578,7 +579,11 @@ func appendMessage(messages []map[string]any, newMessage map[string]any) []map[s
}

func (t Tool) GetAuthTokenHeaderName(resourceMgr tools.SourceProvider) (string, error) {
return "Authorization", nil
source, err := tools.GetCompatibleSource[compatibleSource](resourceMgr, t.Source, t.Name, t.Type)
if err != nil {
return "", err
}
return source.GetAuthTokenHeaderName(), nil
}

func (t Tool) GetParameters() parameters.Parameters {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ type compatibleSource interface {
BigQuerySession() bigqueryds.BigQuerySessionProvider
BigQueryWriteMode() string
UseClientAuthorization() bool
GetAuthTokenHeaderName() string
IsDatasetAllowed(projectID, datasetID string) bool
BigQueryAllowedDatasets() []string
RetrieveClientAndService(tools.AccessToken) (*bigqueryapi.Client, *bigqueryrestapi.Service, error)
Expand Down Expand Up @@ -311,7 +312,11 @@ func (t Tool) RequiresClientAuthorization(resourceMgr tools.SourceProvider) (boo
}

func (t Tool) GetAuthTokenHeaderName(resourceMgr tools.SourceProvider) (string, error) {
return "Authorization", nil
source, err := tools.GetCompatibleSource[compatibleSource](resourceMgr, t.Source, t.Name, t.Type)
if err != nil {
return "", err
}
return source.GetAuthTokenHeaderName(), nil
}

func (t Tool) GetParameters() parameters.Parameters {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.T
type compatibleSource interface {
BigQueryClient() *bigqueryapi.Client
UseClientAuthorization() bool
GetAuthTokenHeaderName() string
IsDatasetAllowed(projectID, datasetID string) bool
BigQueryAllowedDatasets() []string
BigQuerySession() bigqueryds.BigQuerySessionProvider
Expand Down Expand Up @@ -301,7 +302,11 @@ func (t Tool) RequiresClientAuthorization(resourceMgr tools.SourceProvider) (boo
}

func (t Tool) GetAuthTokenHeaderName(resourceMgr tools.SourceProvider) (string, error) {
return "Authorization", nil
source, err := tools.GetCompatibleSource[compatibleSource](resourceMgr, t.Source, t.Name, t.Type)
if err != nil {
return "", err
}
return source.GetAuthTokenHeaderName(), nil
}

func (t Tool) GetParameters() parameters.Parameters {
Expand Down
Loading
Loading