Marquez dataset & job notifications' panel

[pawel-big-lebowski]

Several things can happen within a dataset or a job:

• a new column has been added
• dataset content was truncated 3 days ago
• job is failing since column renamed

As a data engineer I can think of plenty of such scenarios. We do have in Marqeuz all the information to detect it but it is not that easy to visualize them. As a a data engineer, with a limited web UI skills, it is hard for me to make use of this information although I know how valuable it is.

In each dataset/job view we could have something like Notifications where such information can be presented. This would allow other Marquez contributors to implement their own notifications or alerts.

[wslulciuc]

@pawel-big-lebowski Love this idea! Added it to our roadmap. I'd be great to get a proposal written outlining how the web UI would display the notifications and any additional API changes needed to support aggregating the notification events in order to provide this level of transparency (similar to what you've outlined above). Would you be interested in writing the proposal?