This repository was archived by the owner on Oct 16, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 49
This repository was archived by the owner on Oct 16, 2025. It is now read-only.
Correct arguments for gcloud dataflow jobs run #73
Copy link
Copy link
Open
Description
Hi - I get an error with the last line in deploy-data-tokeninzation-solution.sh.
gcloud config set project <project_id>
sh deploy-data-tokeninzation-solution.sh
...
gcloud dataflow jobs run ${jobId} --gcs-location gs://dataflow-templates/latest/Stream_DLP_GCS_Text_to_BigQuery --parameters --region=us-central1,inputFilePattern=gs://${DATA_STORAGE_BUCKET}/CCRecords_1564602825.csv,dlpProjectId=${PROJECT_ID},deidentifyTemplateName=${DEID_TEMPLATE_NAME},inspectTemplateName=${INSPECT_TEMPLATE_NAME},datasetName=${BQ_DATASET_NAME},batchSize=500
ERROR: (gcloud.dataflow.jobs.run) argument --parameters: expected one argument
Usage: gcloud dataflow jobs run JOB_NAME --gcs-location=GCS_LOCATION [optional flags]
optional flags may be --additional-experiments | --dataflow-kms-key |
--disable-public-ips | --enable-streaming-engine |
--help | --max-workers | --network | --num-workers |
--parameters | --region | --service-account-email |
--staging-location | --subnetwork |
--worker-machine-type | --worker-region |
--worker-zone | --zoneIf I switch the args around, pulling region out of parameters, then it works:
gcloud dataflow jobs run ${jobId} --gcs-location gs://dataflow-templates/latest/Stream_DLP_GCS_Text_to_BigQuery --region=us-central1 --parameters=inputFilePattern=gs://${DATA_STORAGE_BUCKET}/CCRecords_1564602825.csv,dlpProjectId=${PROJECT_ID},deidentifyTemplateName=${DEID_TEMPLATE_NAME},inspectTemplateName=${INSPECT_TEMPLATE_NAME},datasetName=${BQ_DATASET_NAME},batchSize=500
createTime: '2021-08-11T11:14:10.713985Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: 2021-08-11_04_14_08-3628396078995258394
location: us-central1
name: demo-dlp-deid-pipeline-20210811-111101
projectId: <project-id>
startTime: '2021-08-11T11:14:10.713985Z'
type: JOB_TYPE_STREAMINGMetadata
Metadata
Assignees
Labels
No labels