Skip to content

RDB Shredder: fix folder discovery at the root of the bucket #704

Open
@chuwy

Description

@chuwy

If user saves the data at the root of the bucket, like:

s3://snowplow-bucket/run=2022-01-28-16-30-00/

Instead of:

s3://snowplow-bucket/shredded/run=2022-01-28-16-30-00/

Our algorithm fails to figure out the bucket is processed already.

Details: https://discourse.snowplowanalytics.com/t/upgrade-shredder-from-0-19-0-to-2-0-0-decoding-failure-in-shredding-job/6261/21

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions