Add the ability to emit custom events

[bvandewe]

What would you like to be added?

It'd be super powerful if CloudShapes would enable users configuring triggers to define an optional custom event to be emitted/published after handling a trigger.

For example, we could easily subscribe CloudShape to any stream (in CloudStreams term) and have it maintain for example a Metrics projection with few custom metrics that are computed in real time whenever handling any event of the stream, right?

What if we'd want to display these metrics in a dashboard somewhere?

Right now, we could "simply" subscribe the dashboard' backend to receive all events that CloudShapes emits already... which is totally awesome BUT which forces the subscriber/backend to explicitly handle CS's event' types/schema and possibly handle a lot of unwanted/noise traffic (as CloudStreams would send ALL events from CloudShapes... right?)

Could CS allow users to define "custom" events (to be emitted) whenever handling a trigger in an incoming event - and by extension, and/or whenever a predefined data path changed in a projection?

Proposal(s):

No response

Alternative(s):

No response

Additional info:

No response

Community Notes

• Please vote by adding a 👍 reaction to the feature to help us prioritize.
• If you are interested to work on this feature, please leave a comment.

[cdavernas]

@JBBianchi WDYT?

[JBBianchi]

Hello @bvandewe ,
Thanks for your interest in CloudShapes!

At first, my instinct was to see this as outside CloudShapes’ scope since it already emits events for every trigger. However, on second thought, I see the nuance—CloudShapes is indeed the source of truth when it comes to changes, especially when dealing with nested state properties.

I’m not entirely convinced about introducing fully user-defined custom events, as that could add complexity and blur the lines of responsibility. That said, I see potential in a middle ground—perhaps CloudShapes could emit a more structured "change event" that explicitly highlights what has changed within a projection. This would allow downstream consumers to react more efficiently without needing to process all CloudShapes events.

Curious to hear your thoughts on this approach—does it align with what you’re looking for? @cdavernas wdyt ?

[bvandewe]

My main concern is that the events currently emitted seem to have their types hardcoded to 'cloud-shapes' (ref), therefore (unless CloudStreams' correlation-based streams could be used to define streams per projectionType/projectionInstance?) any subscriber would need to (be subscribed to cloudshapes events, per type or per source and therefore) handle all events from cloudshapes, even for streams/projections that arent relevant (would see all projections' events while likely only few are relevant to the subscriber).

Maybe a middle ground could be to let the user define a single cloudevent type attribute for the resulting Projection(Type) event that cloud-shapes emits - and not its payload (assuming the payload includes enough insights about the change or the full projection?... maybe an option?)?

Something like emit_type:

name: Order
description: >
  The `Order` projection represents an **order entity**, materialized from CloudEvents. 
  It includes **embedded customer and item data** to ensure real-time consistency and optimized query performance.
schema:
  type: object
indexes:
  - name: store
    fields: [ store ]
    unique: false
    descending: false
    text: false
triggers:
  create:
    - event:
        type: io.petstore.events.order.created.v1
        correlation:
          attribute: subject
      state: '${ .data }'
      emit_type: com.myapp.order.created.v1   # <<< tells cloud-shapes to use this `type` when emitting the projection.created.v1 event
  update:
    - event:
        type: io.petstore.events.order.updated.v1
        correlation:
          attribute: subject
      state: '${ .data }'
      emit_type: com.myapp.order.statechangerequested.v1   # <<< could rename to something more relevant to the subscriber
  delete:
    - event:
        type: io.petstore.events.order.deleted.v1
        correlation:
          attribute: subject
      emit_type: com.myapp.order.cancelled.v1    #
relationships:
  - type: one-to-one
    target: Customer
    key: customerId
    path: customer
  - type: one-to-many
    target: OrderItem
    key: orderId
    path: items

[cdavernas]

therefore (unless CloudStreams' correlation-based streams could be used to define streams per projectionType/projectionInstance?) any subscriber would need to (be subscribed to cloudshapes events, per type or per source and therefore) handle all events from cloudshapes

@bvandewe That is not entirely true, as Cloud Streams allows filtering events, on a per consumer basis. You could therefore decide to filter only to to the relevant Cloud Shapes events.

I'm not a fan of your last suggestion, as Cloud Shapes would anyways have to produce another event with the system's type on top of the configured one. I'd therefore suggest we either provide the feature to produce a fully customizable event upon trigger activation, or none at all.