SealogReader/SealogWriter

[davidpablocohn]

Benefits of a SealogWriter: able to register an event when the GeofenceTransform or QCFilterTransform identifies that we've crossed an EEZ boundary, or some data value has crossed some threshold.

Benefits of a SealogReader: able to use Sealog events to trigger OpenRVDAS mode/config changes using LoggerManagerWriter. Probably other benefits as well.

[webbpinner]

The sealog-server README.md provides a good start on how to submit events and listen to the websocket pub/sub. https://github.com/OceanDataTools/sealog-server/blob/2.x/README.md

[webbpinner]

ok... circling back to this.

Connecting to Sealog and submitting an event is easy... it's defining the contents of the even within the logger_config.yaml file that is giving me trouble.

Here's the example logger_config from logger_manager_writer:

# Read parsed DASRecords from UDP
readers:
  class: UDPReader
  kwargs:
    port: 6224
# Look for lat/lon values in the DASRecords and emit appropriate commands
# when entering/leaving EEZ. Note that EEZ files in GML format can be
# downloaded from https://marineregions.org/eezsearch.php.
transforms:
  - class: GeofenceTransform
    module: logger.transforms.geofence_transform
    kwargs:
      latitude_field_name: s330Latitude,
      longitude_field_name: s330Longitude
      boundary_file_name: /tmp/eez.gml
      leaving_boundary_message: set_active_mode write+influx
      entering_boundary_message: set_active_mode no_write+influx
# Send the messages that we get from geofence to the LoggerManager
writers:
  - class: LoggerManagerWriter
    module: logger.writers.logger_manager_writer
    kwargs:
      database: django
      allowed_prefixes:
        - 'set_active_mode '
        - 'sleep '

So if we want OpenRVDAS to submit an event indicating the mode switch how would we define that?

Here's the json object Sealog is expecting:

{
  event_value: 'OpenRVDAS',
  ts: '2025-02-10T14:37:40.000Z',
  event_author: 'openrvdas',
  event_options: [
    {
      event_option_name: 'status',
      event_option_value: 'mode change'
    },
    {
      event_option_name: 'mode',
      event_option_value: 'no_write+influx'
    }
  ]
}

One idea:

writers:
  - class: LoggerManagerWriter
    module: logger.writers.logger_manager_writer
    kwargs:
      database: django
      allowed_prefixes:
        - "set_active_mode "
        - "sleep "
  - class: SealogWriter
    module: logger.writers.sealog_writer
    kwargs:
      include_regex: "^set_active_mode*"
      event_value: OpenRVDAS
      event_options:
        status: 'mode change'
        mode:
          - match: 'set_active_mode write+influx'
            value:
              mode: 'write+influx'
          - match: 'set_active_mode no_write+influx'
            value:
              mode: 'no_write+influx'

include_regex would be optional.
There could also be an optional exclude_regex kwarg

The event_author would also be optional as if it's not defined in the JSON, Sealog uses the author encoded in the JWT authentication token. ts is also optional as Sealog will use it's system time by default.

Thoughts?

What other use cases can you think of for when OpenRVDAS would want to submit an event?

[davidpablocohn]

Thanks, and greetings from Hobart! I can imagine having QCFilters (or TimeoutWriters) log certain types of errors, geofence triggers (whether or not they're associated with mode changes), But I don't have a sense of what events people log in general, so I'm probably not the best judge. It also seems useful to be able to have the event value include some/all of the string that it receives. Since you're already using a regex to match, maybe it could be include named variables that could then be put into the event_option_value?

…

On Tue, Feb 11, 2025 at 1:50 AM Webb Pinner ***@***.***> wrote: ok... circling back to this. Connecting to Sealog and submitting an event is easy... it's defining the contents of the even within the logger_config.yaml file that is giving me trouble. Here's the example logger_config from logger_manager_writer: # Read parsed DASRecords from UDP readers: class: UDPReader kwargs: port: 6224 # Look for lat/lon values in the DASRecords and emit appropriate commands # when entering/leaving EEZ. Note that EEZ files in GML format can be # downloaded from https://marineregions.org/eezsearch.php. transforms: - class: GeofenceTransform module: logger.transforms.geofence_transform kwargs: latitude_field_name: s330Latitude, longitude_field_name: s330Longitude boundary_file_name: /tmp/eez.gml leaving_boundary_message: set_active_mode write+influx entering_boundary_message: set_active_mode no_write+influx # Send the messages that we get from geofence to the LoggerManager writers: - class: LoggerManagerWriter module: logger.writers.logger_manager_writer kwargs: database: django allowed_prefixes: - 'set_active_mode ' - 'sleep ' So if we want OpenRVDAS to submit an event indicating the mode switch how would we define that? Here's the json object Sealog is expecting: { event_value: 'OpenRVDAS', ts: '2025-02-10T14:37:40.000Z', event_author: 'openrvdas', event_options: [ { event_option_name: 'status', event_option_value: 'mode change' }, { event_option_name: 'mode', event_option_value: 'no_write+influx' } ] } One idea: writers: - class: LoggerManagerWriter module: logger.writers.logger_manager_writer kwargs: database: django allowed_prefixes: - "set_active_mode " - "sleep " - class: SealogWriter module: logger.writers.sealog_writer kwargs: include_regex: "^set_active_mode*" event_value: OpenRVDAS event_options: status: 'mode change' mode: - match: 'set_active_mode write+influx' value: mode: 'write+influx' - match: 'set_active_mode no_write+influx' value: mode: 'no_write+influx' include_regex would be optional. There could also be an optional exclude_regex kwarg The event_author would also be optional as if it's not defined in the JSON, Sealog uses the author encoded in the JWT authentication token. ts is also optional as Sealog will use it's system time by default. Thoughts? What other use cases can you think of for when OpenRVDAS would want to submit an event? — Reply to this email directly, view it on GitHub <#379 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFO7V3XBDODJCZ52T5NOWU32PC4CBAVCNFSM6AAAAABW2YPHSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBYGIZTGNJZGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[webbpinner]

The geofence event (great example) would just be a second instance of the Writer

writers:
  - class: LoggerManagerWriter
    module: logger.writers.logger_manager_writer
    kwargs:
      database: django
      allowed_prefixes:
        - "set_active_mode "
        - "sleep "
  - class: SealogWriter
    module: logger.writers.sealog_writer
    kwargs:
      include_regex: "^set_active_mode*"
      event_value: OpenRVDAS
      event_options:
        status: 'mode change'
        mode:
          - match: 'set_active_mode write+influx'
            value: 'write+influx'
          - match: 'set_active_mode no_write+influx'
            value: 'no_write+influx'
  - class: SealogWriter
    module: logger.writers.sealog_writer
    kwargs:
      include_regex: "^set_active_mode*"
      event_value: EEZ
      event_options:
        status:
          - match: 'set_active_mode write+influx'
            value: 'leaving eez'
          - match: 'set_active_mode no_write+influx'
            value: 'entering eez'

I don't know how I would be able to setup Sealog events for OpenRVDAS mode changes that occur outside of those set by the LoggerManagerWriter. There would need to be something like a LoggerManagerReader???

As for passing through the record as a value (great suggestion)... Maybe just define the reserved keyword 'record' and use that to define when to use the record's value in the output?

i.e. event_value: record or event_options: [{ status: record }]

I think there will also have to be some limitations on when this writer can be used... like only when the record passed to the writer is a string.

[davidpablocohn]

Thanks! I'm mostly trying to figure out the broadest possible scope of when you might want to log an "event." I think the way to pick up events from outside things like a GeofenceTransform->LoggerManagerWriter would be with a CachedDataReader subscribed to "status:[whatever]" messages. Would Sealog be happy with an event like: { event_option_name: 'status', event_option_value: 'mode change' }, { event_option_name: 'mode', event_option_value: 'no_write+influx' }, { event_option_name: 'record', event_option_value: '[whatever the record was]' } If so, the yaml spec for an event could just have an extra option: include_record: true to say when it should include that extra tuple, right? BTW: still off in Tasmania in a campervan, so I'm not going to have a lot of opportunity to fully wrap my head around this until I'm back on the 25th. But would love to have a good long think about it when I do. Thanks! -p

…

On Tue, Feb 11, 2025 at 9:49 PM Webb Pinner ***@***.***> wrote: The geofence event (great example) would just be a second instance of the Writer writers: - class: LoggerManagerWriter module: logger.writers.logger_manager_writer kwargs: database: django allowed_prefixes: - "set_active_mode " - "sleep " - class: SealogWriter module: logger.writers.sealog_writer kwargs: include_regex: "^set_active_mode*" event_value: OpenRVDAS event_options: status: 'mode change' mode: - match: 'set_active_mode write+influx' value: 'write+influx' - match: 'set_active_mode no_write+influx' value: 'no_write+influx' - class: SealogWriter module: logger.writers.sealog_writer kwargs: include_regex: "^set_active_mode*" event_value: EEZ event_options: status: - match: 'set_active_mode write+influx' value: 'leaving eez' - match: 'set_active_mode no_write+influx' value: 'entering eez' I don't know how I would be able to setup Sealog events for OpenRVDAS mode changes that occur outside of those set by the LoggerManagerWriter. There would need to be something like a LoggerManagerReader??? As for passing through the record as a value (great suggestion)... Maybe just define the reserved keyword 'record' and use that to define when to use the record's value in the output? i.e. event_value: record or event_options: [{ status: record }] I think there will also have to be some limitations on when this writer can be used... like only when the record passed to the writer is a string. — Reply to this email directly, view it on GitHub <#379 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFO7V3W365DGQVHFCV73B432PHITNAVCNFSM6AAAAABW2YPHSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJQGQZTSNZVHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[webbpinner]

Thanks for taking time out of your vanping schedule to respond. Yes appending:

{
      event_option_name: 'record',
      event_option_value: '[whatever the record was]'
}

to the event record is still a valid record.

...and, I actually like the include_record: true idea because it would act as a short-hand for event_options: [{ status: record }]. Another great suggestion!