Skip to content

Improve support for asynchronous questions #631

Closed
@cortadocodes

Description

@cortadocodes

Feature request

Use Case

We'd like to be able to "fire-and-forget" a question and then retrieve its events (including results) separately later.

Current state

Questions can be asked:

  • Synchronously
  • Asynchronously (but their results and events can't be easily retrieved)

Proposed Solution

  • Adapt Service.ask to work with BigQuery subscriptions
  • Provide convenient asynchronous method or argument in Child class
  • Test sending events to BigQuery via a push subscription
  • Add a function to retrieve an async question's events with:
    • Any combination of the following:
      • Inputs
        • Remove sender_type part of filter in bigquery answer subscriptions
      • Output values and manifest
      • Output datasets
      • Log messages
      • Monitor messages
    • Output format options:
      • JSON
      • Just events
      • Events and attributes
      • Events attributes, and Pub/Sub metadata
  • Add a CLI command that does the same (not needed for MVP)
  • Add a function that replays events retrieved asynchronously
  • Decide whether bigquery tables should be specified per question (by the parent) or per child (by the child)
    • If bigquery tables are specified per question, automatically delete BigQuery subscriptions once their result event has been received and stored
  • Set up bigquery table for project
  • Update documentation including adding an info box warning that an event store (e.g. BigQuery) needs to be set up for events from asynchronous questions to be usable

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions