Skip to content

Switch to event-driven infrastructure and improve support for asynchronous questions

Choose a tag to compare

View all tags
@github-actions github-actions released this 11 Apr 17:47
0.53.0
6999b22
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Summary

This pull request:

  • Makes the SDK fully event-driven by using a single topic to emit/consume events
  • Majorly refactors the event handler to facilitate asynchronous event retrieval
  • Adds the ability to get and replay events from a BigQuery store

Contents (#632)

IMPORTANT: There are 4 breaking changes.

New features

  • 💥 BREAKING CHANGE: Use single topic per workspace (#639)
  • Add get_events function for retrieving events asynchronously from BigQuery
  • Add EventReplayer class to replay asynchronously-retrieved events
  • Add Manifest.download method

Enhancements

  • Get subscription project name from topic by default
  • Improve asking of asynchronous questions via Child.ask
  • Return download path from Dataset.download
  • Include question UUID in delivery acknowledgement log message
  • Improve handling of invalid events
  • Add datetime and uuid attributes to all events

Fixes

  • Await successful publishing of question messages
  • Fix api_access_endpoint usage in mock_generate_signed_url

Operations

  • Add test BigQuery dataset, cloud function, and IAM roles to terraform config
  • Switch to reusable workflows where possible

Dependencies

  • Add google-cloud-bigquery
  • Upgrade coolname
  • Add db-dtypes for converting bigquery rows to dataframes

Refactoring

  • 💥 BREAKING CHANGE: Rename x.received_messages to x.received_events
  • 💥 BREAKING CHANGE: Rename record_messages parameters to record_events
  • 💥 BREAKING CHANGE: Update ChildEmulator to use event* instead of message*
  • Factor out making minimal dictionary
  • Factor out creating push subscription
  • Factor out emitting question event in Service.ask
  • Factor out event handlers and related logic from OrderedMessageHandler into new AbstractEventHandler
  • Move validation module into octue.cloud.events subpackage
  • Rename OrderedMessageHandler to GoogleCloudPubSubEventHandler
  • Rename "message" to "event" in event handler classes
  • Rename GooglePubSubHandler to GoogleCloudPubSubHandler

Chores

  • Update licence year to 2024

Testing

  • Simplify various tests

Upgrade instructions

  • Update all services in your services network to this version of octue or later (0.53.0+)
  • Replace any usages of the received_messages methods with received_events
  • Replace any usages of the record_messages parameters with record_events
  • Replace the word message with event in usages of ChildEmulator methods (apart from in the case of monitor_message)
Assets 2
Loading