Merging code for 'Utilizing Cloud Logging and Cloud Run to create Event Driven Data Pipelines'#283
Merging code for 'Utilizing Cloud Logging and Cloud Run to create Event Driven Data Pipelines'#283ghost wants to merge 17 commits intoGoogleCloudPlatform:masterfrom
Conversation
There was a problem hiding this comment.
Remove this entire file
tools/cloud_run/event_driven_bq_ingest/src/main/java/com/example/cloudrun/BQAccessor.java
Outdated
Show resolved
Hide resolved
tools/cloud_run/event_driven_bq_ingest/src/main/java/com/example/cloudrun/GCSAccessor.java
Outdated
Show resolved
Hide resolved
| @@ -0,0 +1,64 @@ | |||
| /* | |||
There was a problem hiding this comment.
My mistake; I see this is needed to map the message into a JSON object
There was a problem hiding this comment.
This file would be better named: BigQueryMetadataAuditLog
There was a problem hiding this comment.
If you're going to have a class like this, you should have it also store bq job notifications which come in through pubsub.
There was a problem hiding this comment.
Why parse bq job notifications differently than gcs notifications? Unify with one parser that handles both
There was a problem hiding this comment.
Remove if this is not needed
There was a problem hiding this comment.
This does not mean the job was successful, only that that job completed.
You must check the status of the job as follows:
if(job.isDone() && job.getStatus().getError() == null)
Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
…le/cloudrun/BQAccessor.java Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
…le/cloudrun/GCSAccessor.java Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
Co-authored-by: Daniel De Leo <danieldeleo@users.noreply.github.com>
Merging changes for parsing pubsub messages efficiently
Removed extra dependencies from pom and reformat code
2 participants
@danieldeleo