-
Notifications
You must be signed in to change notification settings - Fork 30
Open
Description
When using regexp it does not pull the latest version of the file. Issues persists no matter if bucket versioning is enabled or not. Would it be possible to default to using the latest file version when using regexp? Or providing a new resource parameter to enable that behavior?
For example, given a resource
- name: saved_cluster_env_files
type: gcs
source:
json_key: ((gcs-service-account-key))
bucket: some-bucket
regexp: sub-bucket/cluster_env_files(.*).tar.gz
the first job in my pipeline pushes the resource called cluster_env_files_MyBranchName.tar.gz. Subsequent jobs then pull that same resource. However, if the pipeline get's re-flown or the first job is re-run, the following jobs are matching the correct filename, but pulling down a previous version of it. This causes those jobs to fail.
Metadata
Metadata
Assignees
Labels
No labels