Skip to content

regexp not using latest version #47

@kalensk

Description

@kalensk

When using regexp it does not pull the latest version of the file. Issues persists no matter if bucket versioning is enabled or not. Would it be possible to default to using the latest file version when using regexp? Or providing a new resource parameter to enable that behavior?

For example, given a resource

  - name: saved_cluster_env_files
    type: gcs
    source:
      json_key: ((gcs-service-account-key))
      bucket: some-bucket
      regexp: sub-bucket/cluster_env_files(.*).tar.gz

the first job in my pipeline pushes the resource called cluster_env_files_MyBranchName.tar.gz. Subsequent jobs then pull that same resource. However, if the pipeline get's re-flown or the first job is re-run, the following jobs are matching the correct filename, but pulling down a previous version of it. This causes those jobs to fail.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions