BigQuery Data Transfer Config Parameter type mismatch

[cameronwilson99]

Describe what happened

Getting this issue recently where the DataTransferConfig won't update the actual resource, and will only successfully pass a pulumi up command every other time it is run. Even the desired change it wants to make changes.

End up getting this error: googleapi: Error 400: Parameter type required for assets: LIST, Parameter type provided in transfer config: string_value.

I still got this issue even when upgrading the GCP provider to the latest (9.9.0). Perhaps I'm missing something?

Here is the output of two back-to-back pulumi up commands:

<>: ~/git/infra/pulumi/global (main)$ pulumi up --suppress-outputs --diff
Previewing update (global)

View Live: <>

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
    ~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
        [id=<>]
        [urn=<>]
        [provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
      ~ params: {
          ~ connector.authentication.password: [secret] => [secret]
        }
Resources:
    ~ 1 to update
    280 unchanged
Do you want to perform this update? yes
Updating (global)

View Live: <>

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
    ~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
        [id=<>]
        [urn=<>]
        [provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
      ~ params: {
          ~ connector.authentication.password: [secret] => [secret]
        }
error: 1 error occurred:
	* updating <>: 1 error occurred:
	* Error updating Config "<>": googleapi: Error 400: Parameter type required for assets: LIST, Parameter type provided in transfer config: string_value. Config name: <>

error: update failed
Resources:
    280 unchanged
    <{%fg 1%}>1 errored<{%reset%}>

Duration: 3s
^[[A%                                                                                                                                                                                             <>: ~/git/infra/pulumi/global (main)$ pulumi up --suppress-outputs --diff
Previewing update (global)

View Live: <>

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
    ~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
        [id=<>]
        [urn=<>]
        [provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
        dataSourceId        : "postgresql"
        destinationDatasetId: "<>"
        displayName         : "production-db-to-bigquery-live-sync"
        emailPreferences    : {
            enableFailureEmail: true
        }
        location            : "us-west3"
        params              : {
            assets                           : "<>"
            connector.authentication.password: [secret]
            connector.authentication.username: "<>"
            connector.database               : "<>"
            connector.endpoint.host          : "<>"
            connector.endpoint.port          : "<>"
            connector.networkAttachment      : "<>"
            connector.tls.mode               : "DISABLE"
        }
        project             : "<>"
        schedule            : "every 15 minutes"
        scheduleOptions     : {
            startTime : "2025-12-05T00:06:00Z"
        }
        serviceAccountName  : ""
Resources:
    ~ 1 to update
    280 unchanged
Do you want to perform this update? yes
Updating (global)

View Live: <>

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
    ~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
        [id=<>]
        [urn=<>]
        [provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
        dataSourceId        : "postgresql"
        destinationDatasetId: "<>"
        displayName         : "production-db-to-bigquery-live-sync"
        emailPreferences    : {
            enableFailureEmail: true
        }
        location            : "<>"
        params              : {
            assets                           : "<>"
            connector.authentication.password: [secret]
            connector.authentication.username: "<>"
            connector.database               : "<>"
            connector.endpoint.host          : "<>"
            connector.endpoint.port          : "<>"
            connector.networkAttachment      : "<>"
            connector.tls.mode               : "DISABLE"
        }
        project             : "<>"
        schedule            : "every 15 minutes"
        scheduleOptions     : {
            startTime : "2025-12-05T00:06:00Z"
        }
        serviceAccountName  : ""
Resources:     
    ~ 1 updated

So as you can see, it first fails with that error I pointed out, then right after succeeds, with an entirely different diff. It flip-flops like that, without any code changes.

We didn't have this issue until I did a pulumi refresh around the middle of December.

Here is my configuration:

const assets = `[${syncedTables.map(x => `Production/public/${x}`).join(" ")}]`;

// Data transfer config for syncing the production database to BigQuery
export const production_db_to_bigquery_live_sync = new gcp.bigquery.DataTransferConfig("production-db-to-bigquery-live-sync", {
  dataSourceId: "postgresql",
  destinationDatasetId: productionDbDataset.datasetId,
  displayName: "production-db-to-bigquery-live-sync",
  emailPreferences: {
    enableFailureEmail: true,
  },
  location: "us-west3",
  params: {
    assets,
    "connector.authentication.password": config.requireSecret("password"),
    "connector.authentication.username": config.require("username"),
    "connector.database": config.require("database"),
    "connector.endpoint.host": config.require("host"),
    "connector.endpoint.port": config.require("port"),
    "connector.networkAttachment": config.require("networkAttachment"),
    "connector.tls.mode": "DISABLE",
  },
  project: projectId,
  schedule: "every 15 minutes",
  scheduleOptions: {
    startTime: "2025-12-05T00:06:00Z",
  },
}, {
  protect: true,
});

Sample program

const assets = [${syncedTables.map(x => Production/public/${x}).join(" ")}];

// Data transfer config for syncing the production database to BigQuery
export const production_db_to_bigquery_live_sync = new gcp.bigquery.DataTransferConfig("production-db-to-bigquery-live-sync", {
dataSourceId: "postgresql",
destinationDatasetId: productionDbDataset.datasetId,
displayName: "production-db-to-bigquery-live-sync",
emailPreferences: {
enableFailureEmail: true,
},
location: "us-west3",
params: {
assets,
"connector.authentication.password": config.requireSecret("password"),
"connector.authentication.username": config.require("username"),
"connector.database": config.require("database"),
"connector.endpoint.host": config.require("host"),
"connector.endpoint.port": config.require("port"),
"connector.networkAttachment": config.require("networkAttachment"),
"connector.tls.mode": "DISABLE",
},
project: projectId,
schedule: "every 15 minutes",
scheduleOptions: {
startTime: "2025-12-05T00:06:00Z",
},
}, {
protect: true,
});

Log output

<>: ~/git/infra/pulumi/global (main)$ pulumi up --suppress-outputs --diff
Previewing update (global)

View Live: <>

pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
[id=<>]
[urn=<>]
[provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
~ params: {
~ connector.authentication.password: [secret] => [secret]
}
Resources:
~ 1 to update
280 unchanged
Do you want to perform this update? yes
Updating (global)

View Live: <>

pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
[id=<>]
[urn=<>]
[provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
~ params: {
~ connector.authentication.password: [secret] => [secret]
}
error: 1 error occurred:
* updating <>: 1 error occurred:
* Error updating Config "<>": googleapi: Error 400: Parameter type required for assets: LIST, Parameter type provided in transfer config: string_value. Config name: <>

error: update failed
Resources:
280 unchanged
<{%fg 1%}>1 errored<{%reset%}>

Duration: 3s
^[[A% <>: ~/git/infra/pulumi/global (main)$ pulumi up --suppress-outputs --diff
Previewing update (global)

View Live: <>

pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
[id=<>]
[urn=<>]
[provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
dataSourceId : "postgresql"
destinationDatasetId: "<>"
displayName : "production-db-to-bigquery-live-sync"
emailPreferences : {
enableFailureEmail: true
}
location : "us-west3"
params : {
assets : "<>"
connector.authentication.password: [secret]
connector.authentication.username: "<>"
connector.database : "<>"
connector.endpoint.host : "<>"
connector.endpoint.port : "<>"
connector.networkAttachment : "<>"
connector.tls.mode : "DISABLE"
}
project : "<>"
schedule : "every 15 minutes"
scheduleOptions : {
startTime : "2025-12-05T00:06:00Z"
}
serviceAccountName : ""
Resources:
~ 1 to update
280 unchanged
Do you want to perform this update? yes
Updating (global)

View Live: <>

pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:global::global::pulumi:pulumi:Stack::global-global]
~ gcp:bigquery/dataTransferConfig:DataTransferConfig: (update) 🔒
[id=<>]
[urn=<>]
[provider=urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0::75483197-a9ad-4a16-956c-39c860f8a735]
dataSourceId : "postgresql"
destinationDatasetId: "<>"
displayName : "production-db-to-bigquery-live-sync"
emailPreferences : {
enableFailureEmail: true
}
location : "<>"
params : {
assets : "<>"
connector.authentication.password: [secret]
connector.authentication.username: "<>"
connector.database : "<>"
connector.endpoint.host : "<>"
connector.endpoint.port : "<>"
connector.networkAttachment : "<>"
connector.tls.mode : "DISABLE"
}
project : "<>"
schedule : "every 15 minutes"
scheduleOptions : {
startTime : "2025-12-05T00:06:00Z"
}
serviceAccountName : ""
Resources:
~ 1 updated

Affected Resource(s)

const assets = [${syncedTables.map(x => Production/public/${x}).join(" ")}];

// Data transfer config for syncing the production database to BigQuery
export const production_db_to_bigquery_live_sync = new gcp.bigquery.DataTransferConfig("production-db-to-bigquery-live-sync", {
dataSourceId: "postgresql",
destinationDatasetId: productionDbDataset.datasetId,
displayName: "production-db-to-bigquery-live-sync",
emailPreferences: {
enableFailureEmail: true,
},
location: "us-west3",
params: {
assets,
"connector.authentication.password": config.requireSecret("password"),
"connector.authentication.username": config.require("username"),
"connector.database": config.require("database"),
"connector.endpoint.host": config.require("host"),
"connector.endpoint.port": config.require("port"),
"connector.networkAttachment": config.require("networkAttachment"),
"connector.tls.mode": "DISABLE",
},
project: projectId,
schedule: "every 15 minutes",
scheduleOptions: {
startTime: "2025-12-05T00:06:00Z",
},
}, {
protect: true,
});

Output of pulumi about

cameronwilson: ~/git/infra/pulumi/global (main)$ pulumi about
CLI
Version 3.212.0
Go Version go1.25.5
Go Compiler gc

Plugins
KIND NAME VERSION
language nodejs 3.212.0-dev.0

Host
OS darwin
Version 26.1
Arch arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v25.2.1' packagemanager='npm' packagemanagerVersion='11.6.2'

Current Stack: nectarhr/global/global

TYPE URN
pulumi:pulumi:Stack urn:pulumi:global::global::pulumi:pulumi:Stack::global-global
pulumi:providers:gcp urn:pulumi:global::global::pulumi:providers:gcp::default_7_27_0
pulumi:providers:gcp urn:pulumi:global::global::pulumi:providers:gcp::gcp-v9
pulumi:providers:gcp urn:pulumi:global::global::pulumi:providers:gcp::default_7_38_0
pulumi:providers:gcp urn:pulumi:global::global::pulumi:providers:gcp::default_9_9_0
pulumi:providers:gcp urn:pulumi:global::global::pulumi:providers:gcp::default_8_0_0

Found no pending operations associated with nectarhr/global

Backend
Name pulumi.com
URL https://app.pulumi.com/cameron.wilson1
User cameron.wilson1
Organizations cameron.wilson1, nectarhr, cameron-wilson1-org
Token type personal

Dependencies:
NAME VERSION
typescript 5.0.4
@pulumi/gcp-v9 9.9.0
@pulumi/pulumi 3.113.3
@types/node 18.19.33
@google-cloud/monitoring 4.1.0
@pulumi/gcp 7.27.0

Pulumi locates its logs in /var/folders/64/swg8473108ngjgkj0l0fq7p80000gp/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[pgavlin]

Sorry to hear that you're hitting this. I'm wondering if the update is returning some partial state which in turn influences the diff in the next pulumi up.

It would be helpful to have the state of the transfer config before and after the first pulumi up. Is that something you might be able to share? You should be able to find it under the resources tab in the cloud console or in the statefile returned by pulumi stack export.