Merge branch 'main' into prebuilt-and-banner

Author: dishaprakash

UPGRADING.md

@@ -0,0 +1,113 @@
+# Upgrading to MCP Toolbox for Databases v1.0.0
+
+Welcome to the v1.0.0 release of the MCP Toolbox for Databases! 
+
+This release stabilizes our core APIs and standardizes our protocol alignments.
+As part of this milestone, we have introduced several breaking changes and
+deprecations that require updates to your configuration and code.
+
+**📖 New Versioning Policy**
+We have officially published our [Versioning Policy](https://googleapis.github.io/genai-toolbox/dev/about/versioning/). Moving forward, we follow standard versioning conventions to classify updates:
+* **Major (vX.0.0):** Breaking changes requiring manual updates.
+* **Minor (v1.X.0):** New, backward-compatible features and deprecation notices.
+* **Patch (v1.0.X):** Backward-compatible bug fixes and security patches.
+
+This guide outlines what has changed and the steps you need to take to upgrade.
+
+## 🚨 Breaking Changes (Action Required)
+
+### 1. Endpoint Transition: `/api` disabled by default
+The legacy `/api` endpoint for the native Toolbox protocol is now disabled by default. All official SDKs have been updated to use the `/mcp` endpoint, which aligns with the standard Model Context Protocol (MCP) specification. 
+
+If you still require the legacy `/api` endpoint, you must explicitly activate it using a new command-line flag.
+
+* **Usage:** `./toolbox --enable-api`
+* **Migration:** You must update all custom implementations to use the `/mcp`
+  endpoint exclusively, as the `/api` endpoint is now deprecated. If your workflow  
+  relied on a non-standard feature that is missing from the new implementation, please submit a
+  feature request on our [GitHub Issues page](https://github.com/googleapis/genai-toolbox/issues).
+* **UI Dependency:** Until the UI is officially migrated, it still requires the API to function. You must run the toolbox with both flags: `./toolbox --ui --enable-api`.
+
+### 2. Strict Tool Naming Validation (SEP986)
+Tool names are now strictly validated against [ModelContextProtocol SEP986 guidelines](https://github.com/alexhancock/modelcontextprotocol/blob/main/docs/specification/draft/server/tools.mdx#tool-names) prior to MCP initialization.
+* **Migration:** Ensure all your tool names **only** contain alphanumeric characters, hyphens (`-`), underscores (`_`), and periods (`.`). Any other special characters will cause initialization to fail.
+
+### 3. Removed CLI Flags
+The legacy snake_case flag `--tools_file` has been completely removed.
+* **Migration:** Update your deployment scripts to use `--config` instead.
+
+### 4. Singular `kind` Values in Configuration
+_(This step applies only if you are currently using the new flat format.)_
+
+All primitive kind fields in configuration files have been updated to use singular nouns instead of plural. For example, `kind: sources` is now `kind: source`, and `kind: tools` is now `kind: tool`.
+
+* **Migration:** Update your configuration files to use the singular form for all `kind`
+values. _(Note: If you transitioned to the flat format using the `./toolbox migrate` command, this step was handled automatically.)_
+
+
+### 5. Configuration Schema: `authSources` renamed
+The `authSources` field is no longer supported in configuration files.
+* **Migration:** Rename all instances of `authSources` to `authService` in your
+  configuration files.
+
+### 6. CloudSQL for SQL Server: `ipAddress` removed
+The `ipAddress` field for the CloudSQL for SQL Server source was redundant and has been removed.
+* **Migration:** Remove the `ipAddress` field from your CloudSQL for SQL Server configurations.
+
+
+## ⚠️ Deprecations & Modernization
+
+### 1. Flat Configuration Format Introduced
+We have introduced a new, streamlined "flat" format for configuration files. While the older nested format is still supported for now, **all new features will only be added to the flat format.**
+
+**Schema Restructuring (`kind` vs. `type`):**
+Along with the flat format, the configuration schema has been reorganized. The
+old `kind` field (which specified the specific primitive types, like
+`alloydb-postgres`) has been renamed to `type`. The `kind` field is now strictly
+used to declare the core primitive of the block (e.g., `source` or `tool`).
+
+**Example of the new flat format:**
+
+```yaml
+kind: source
+name: my-source
+type: alloydb-postgres
+project: my-project
+region: my-region
+instance: my-instance
+---
+kind: tool
+name: my-simple-tool
+type: postgres-execute-sql
+source: my-source
+description: this is a tool that executes the sql provided.
+```
+
+**Migration:**
+
+You can automatically migrate your existing nested configurations to the new flat format using the CLI. Run the following command:
+
+```Bash
+./toolbox migrate --config <path-to-your-config>
+```
+_Note: You can also use the `--configs` or `--config-folder` flags with this command._
+
+### 2. Deprecated CLI Flags
+The following CLI flags are deprecated and will be removed in a future release. Please update your scripts:
+
+* `--tools-file` ➡️ Use `--config`
+* `--tools-files` ➡️ Use `--configs`
+* `--tools-folder` ➡️ Use `--config-folder`
+
+## 💡 Other Notable Updates
+* **Enhanced Error Handling:** Errors are now strictly categorized between Agent Errors (allowing the LLM to self-correct) and Client/Server Errors (which signal a hard stop).
+
+* **Telemetry Updates:** The /mcp endpoint telemetry has been revised to fully comply with the [OpenTelemetry semantic conventions for MCP](https://opentelemetry.io/docs/specs/semconv/gen-ai/mcp/).
+
+* **MCP Authorization Support:** The Model Context Protocol's [authorization specification](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization) is now fully supported.
+
+* **Database Name Validation:** Removed the "required field" validation for the database name in CloudSQL for MySQL and generic MySQL sources.
+
+* **Prebuilt Tools:** Toolsets have been resized for better performance.
+## 📚 Documentation Moved
+Our official documentation has a new home! Please update your bookmarks to [mcp-toolbox.dev](http://mcp-toolbox.dev).

cmd/internal/config.go

@@ -27,6 +27,7 @@ import (
 
 	"github.com/goccy/go-yaml"
 	"github.com/google/go-cmp/cmp"
+	"github.com/googleapis/genai-toolbox/internal/auth/generic"
 	"github.com/googleapis/genai-toolbox/internal/server"
 )
 
@@ -309,6 +310,18 @@ func mergeConfigs(files ...Config) (Config, error) {
 		return Config{}, fmt.Errorf("resource conflicts detected:\n  - %s\n\nPlease ensure each source, authService, tool, toolset and prompt has a unique name across all files", strings.Join(conflicts, "\n  - "))
 	}
 
+	// Ensure only one authService has mcpEnabled = true
+	var mcpEnabledAuthServers []string
+	for name, authService := range merged.AuthServices {
+		// Only generic type has McpEnabled right now
+		if genericService, ok := authService.(generic.Config); ok && genericService.McpEnabled {
+			mcpEnabledAuthServers = append(mcpEnabledAuthServers, name)
+		}
+	}
+	if len(mcpEnabledAuthServers) > 1 {
+		return Config{}, fmt.Errorf("multiple authServices with mcpEnabled=true detected: %s. Only one MCP authorization server is currently supported", strings.Join(mcpEnabledAuthServers, ", "))
+	}
+
 	return merged, nil
 }
 

cmd/internal/config_test.go

@@ -20,6 +20,7 @@ import (
 	"testing"
 
 	"github.com/google/go-cmp/cmp"
+	"github.com/googleapis/genai-toolbox/internal/auth/generic"
 	"github.com/googleapis/genai-toolbox/internal/auth/google"
 	"github.com/googleapis/genai-toolbox/internal/embeddingmodels/gemini"
 	"github.com/googleapis/genai-toolbox/internal/prebuiltconfigs"
@@ -616,38 +617,48 @@ func TestParseConfig(t *testing.T) {
 			type: google
 			clientId: testing-id
 ---
-			kind: embeddingModel
-			name: gemini-model
-			type: gemini
-			model: gemini-embedding-001
-			apiKey: some-key
-			dimension: 768
+      kind: authService
+      name: my-generic-auth
+      type: generic
+      audience: testings
+      authorizationServer: https://testings
+      mcpEnabled: true
+      scopesRequired:
+        - read:files
+        - write:files
 ---
-			kind: tool
-			name: example_tool
-			type: postgres-sql
-			source: my-pg-instance
-			description: some description
-			statement: |
-				SELECT * FROM SQL_STATEMENT;
-			parameters:
-			- name: country
-			  type: string
-			  description: some description
+      kind: embeddingModel
+      name: gemini-model
+      type: gemini
+      model: gemini-embedding-001
+      apiKey: some-key
+      dimension: 768
 ---
-			kind: toolset
-			name: example_toolset
-			tools:
-			- example_tool
+      kind: tool
+      name: example_tool
+      type: postgres-sql
+      source: my-pg-instance
+      description: some description
+      statement: |
+        SELECT * FROM SQL_STATEMENT;
+      parameters:
+      - name: country
+        type: string
+        description: some description
 ---
-			kind: prompt
-			name: code_review
-			description: ask llm to analyze code quality
-			messages:
-			- content: "please review the following code for quality: {{.code}}"
-			arguments:
-			- name: code
-			  description: the code to review
+      kind: toolset
+      name: example_toolset
+      tools:
+      - example_tool
+---
+      kind: prompt
+      name: code_review
+      description: ask llm to analyze code quality
+      messages:
+      - content: "please review the following code for quality: {{.code}}"
+      arguments:
+      - name: code
+        description: the code to review
 			`,
 			wantConfig: Config{
 				Sources: server.SourceConfigs{
@@ -669,6 +680,14 @@ func TestParseConfig(t *testing.T) {
 						Type:     google.AuthServiceType,
 						ClientID: "testing-id",
 					},
+					"my-generic-auth": generic.Config{
+						Name:                "my-generic-auth",
+						Type:                generic.AuthServiceType,
+						Audience:            "testings",
+						McpEnabled:          true,
+						AuthorizationServer: "https://testings",
+						ScopesRequired:      []string{"read:files", "write:files"},
+					},
 				},
 				EmbeddingModels: server.EmbeddingModelConfigs{
 					"gemini-model": gemini.Config{
@@ -2029,12 +2048,19 @@ func TestMergeConfigs(t *testing.T) {
 		Sources: server.SourceConfigs{"source1": httpsrc.Config{Name: "source1"}},
 		Tools:   server.ToolConfigs{"tool2": http.Config{Name: "tool2"}},
 	}
+	fileMcp1 := Config{
+		AuthServices: server.AuthServiceConfigs{"generic1": generic.Config{Name: "generic1", McpEnabled: true}},
+	}
+	fileMcp2 := Config{
+		AuthServices: server.AuthServiceConfigs{"generic2": generic.Config{Name: "generic2", McpEnabled: true}},
+	}
 
 	testCases := []struct {
-		name    string
-		files   []Config
-		want    Config
-		wantErr bool
+		name      string
+		files     []Config
+		want      Config
+		wantErr   bool
+		errString string
 	}{
 		{
 			name:  "merge two distinct files",
@@ -2054,6 +2080,12 @@ func TestMergeConfigs(t *testing.T) {
 			files:   []Config{file1, file2, fileWithConflicts},
 			wantErr: true,
 		},
+		{
+			name:      "merge multiple mcp enabled generic",
+			files:     []Config{fileMcp1, fileMcp2},
+			wantErr:   true,
+			errString: "multiple authServices with mcpEnabled=true detected",
+		},
 		{
 			name:  "merge single file",
 			files: []Config{file1},
@@ -2094,7 +2126,9 @@ func TestMergeConfigs(t *testing.T) {
 				if err == nil {
 					t.Fatal("expected an error for conflicting files but got none")
 				}
-				if !strings.Contains(err.Error(), "resource conflicts detected") {
+				if tc.errString != "" && !strings.Contains(err.Error(), tc.errString) {
+					t.Errorf("expected error %q, but got: %v", tc.errString, err)
+				} else if tc.errString == "" && !strings.Contains(err.Error(), "resource conflicts detected") {
 					t.Errorf("expected conflict error, but got: %v", err)
 				}
 			}

docs/en/documentation/configuration/authentication/_index.md

@@ -304,4 +304,4 @@ func main() {
 }
 ```
 
-## Kinds of Auth Services
+## Types of Auth Services

docs/en/documentation/configuration/embedding-models/_index.md

@@ -103,4 +103,4 @@ parameters:
     embeddedBy: gemini-model # refers to the name of a defined embedding model
 ```
 
-## Kinds of Embedding Models
+## Types of Embedding Models

docs/en/documentation/configuration/prompts/_index.md

@@ -78,4 +78,4 @@ The workflow is as follows:
 5. **Response:** This completed prompt is then sent to the Gemini model, and the
    model's response is displayed back to you in the CLI.
 
-## Kinds of prompts
+## Types of prompts

docs/en/documentation/configuration/sources/_index.md

@@ -33,4 +33,4 @@ to connect to the database and execute the tool.
 
 ## Available Sources
 
-To see all supported sources and the specific tools they unlock, explore the full list of our [Integrations](../../../integrations/_index.md).
+To see all supported sources and the specific tools they unlock, explore the full list of our [Integrations](../../../integrations/_index.md).

docs/en/resources/authServices/generic.md

@@ -0,0 +1,67 @@
+---
+title: "Generic OIDC Auth"
+type: docs
+weight: 2
+description: >
+  Use a Generic OpenID Connect (OIDC) provider for OAuth 2.0 flow and token
+  lifecycle.
+---
+
+## Getting Started
+
+The Generic Auth Service allows you to integrate with any OpenID Connect (OIDC)
+compliant identity provider (IDP). It discovers the JWKS (JSON Web Key Set) URL
+either through the provider's `/.well-known/openid-configuration` endpoint or
+directly via the provided `authorizationServer`.
+
+To configure this auth service, you need to provide the `audience` (typically
+your client ID or the intended audience for the token), the
+`authorizationServer` of your identity provider, and optionally a list of
+`scopesRequired` that must be present in the token's claims.
+
+## Behavior
+
+### Token Validation
+
+When a request is received, the service will:
+
+1. Extract the token from the `<name>_token` header (e.g.,
+   `my-generic-auth_token`).
+2. Fetch the JWKS from the configured `authorizationServer` (caching it in the
+   background) to verify the token's signature.
+3. Validate that the token is not expired and its signature is valid.
+4. Verify that the `aud` (audience) claim matches the configured `audience`.
+   claim contains all required scopes.
+5. Return the validated claims to be used for [Authenticated
+   Parameters][auth-params] or [Authorized Invocations][auth-invoke].
+
+[auth-invoke]: ../tools/#authorized-invocations
+[auth-params]: ../tools/#authenticated-parameters
+
+## Example
+
+```yaml
+kind: authServices
+name: my-generic-auth
+type: generic
+audience: ${YOUR_OIDC_AUDIENCE}
+authorizationServer: https://your-idp.example.com
+mcpEnabled: false
+scopesRequired:
+  - read
+  - write
+```
+
+{{< notice tip >}} Use environment variable replacement with the format
+${ENV_NAME} instead of hardcoding your secrets into the configuration file.
+{{< /notice >}}
+
+## Reference
+
+| **field**           | **type** | **required** | **description**                                                                                                                                               |
+| ------------------- | :------: | :----------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| type                |  string  |     true     | Must be "generic".                                                                                                                                            |
+| audience            |  string  |     true     | The expected audience (`aud` claim) in the JWT token. This ensures the token was minted specifically for your application.                                    |
+| authorizationServer |  string  |     true     | The base URL of your OIDC provider. The service will append `/.well-known/openid-configuration` to discover the JWKS URI. HTTP is allowed but logs a warning. |
+| mcpEnabled          |   bool   |    false     | Indicates if MCP endpoint authentication should be applied. Defaults to false.                                                                                |
+| scopesRequired      | []string |    false     | A list of required scopes that must be present in the token's `scope` claim to be considered valid.                                                           |