Skip to content

Add GCS Object Sensor #1218

@terzioglub

Description

@terzioglub

Description

S3 package has a KeySensor implementation (pkg/s3/operator.go) that allows checking if objects exist in S3 buckets, but GCS package is missing the equivalent functionality. This creates feature parity gap between AWS S3 and Google Cloud Storage support in Bruin.

For more details check the documenation : https://getbruin.com/docs/bruin/platforms/s3.html

Implementation

Create pkg/gcs/operator.go with ObjectSensor (or KeySensor) following the same pattern as S3.

Required functionality:

  • Check if GCS object exists at specified path
  • Support bucket_name and object_key parameters ( feel free to recommend different parameter names)
  • Handle GCS authentication and permissions
  • Return appropriate errors for missing objects/access issues

Example Usage

GCS Sensor Asset (assets/wait_for_data.asset.yml):

/* @bruin
name: "wait_for_file"
type: gcs.sensor.key_sensor
connection: gcs-default
parameters:
  bucket_name: "my-data-bucket"
  bucket_key: "path/to/expected/file.csv"

After implementation:

# Wait for GCS object to exist
bruin run wait_for_daily_export 

Acceptance Criteria

  • pkg/gcs/operator.go created with ObjectSensor implementation
  • Sensor checks GCS object existence using bucket_name and object_key parameters
  • Follow same interface pattern as S3 KeySensor
  • Handle GCS authentication via existing GCS client
  • Error handling for missing objects, permission issues, invalid buckets
  • Test file pkg/gcs/operator_test.go with sensor functionality tests

Testing

# Test GCS sensor
bruin run   gcs-sensor-test  

also run it with  --sensor-mode wait flag 

### Reference Implementation
- `pkg/s3/operator.go` - S3 KeySensor implementation to mirror

Metadata

Metadata

Assignees

No one assigned

    Labels

    goPull requests that update go code

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions