Skip to content

new trait supports-collect-measured #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions yeps/yep-314.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
yep: 314
title: supports-collect-measured
author: Blaise Thompson <[email protected]>
status: draft
tags: trait
post-history: 2023-09-28
---

# Abstract

This YEP introduces a message "collect_measured", a way to grab multiple measurements from a sensor simultaniously.

# Table of Contents

[TOC]

# Motivation

Some experiments incorporate sensors that are very fast or very slow compared to other pieces of hardware being controlled. In such cases it might be natural to let the sensor run asyncronously with other hardware, simply recording each time there is a new sensor reading. Unfortunately, the design of the "get_measured" message defined by is-sensor does not make asyncronus acquistion easy. Clients must ensure that they are polling quickly in order to ensure they don't miss measurements. In extreme cases, sensors are so fast that client polling is simply not practical.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure I am understanding the application, this is most useful for when sensors are very fast? Does this trait also offer advantages when sensors are very slow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's most useful for very fast things.

For very slow things it's also great, because you just don't have to poll.

It's useful for any measurement that's fundamentally asynchronous with your central datastream.


This trait extends is-sensor with exactly one additional message, "collect_measured". This message allows the client to request a whole set of cached measurements from the daemon at once. In this way, clients can take their time and simply "catch up" with all the measurements that a sensor has made all at once.

# Specification

This trait requires is-sensor: see [YEP-302](https://yeps.yaq.fyi/302/).

This trait adds the method `collect_measured`. This method returns an avro array of arrays, where each inner array contains exactly two items: a unix timestamp float and the associated measurement as an avro map. Bluesky users can think of this as a list of readings.

`collect_measured` accepts one optional argument, `measurement_id`, an integer with default of `null`. If a client provides this argument, the daemon will return only measurements since that id, inclusive. Because each measurement mapping already contains `measurement_id` as specified by the is-sensor trait, there is no ambiguity if clients need to cross-reference to ids. Daemons must account for overflow of `measurement_id`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the standard behavior of collect_measured with no argument? All measurements are returned? The last measurement is returned?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea is that all the messages in the cache would be returned. I suppose we could have a second optional argument of max_measurements with default of zero meaning "all".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider allowing negative integers as an argument; e.g. collect_measured(-2) will grab the last two measurements?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would work, as an alternative to my suggestion of a max_measurements argument above...

I think I like it.


There is no cache size gauarantee. Daemons might have dynamic or hard-coded cache sizes, and they might be smart enough to recognize system constraints in allocating caches. Some peripherals may offer caching outside of memory. Certain daemons might support configuration or messages for configuring caching behavior, but there will be no project-wide convention. Ultimately this trait is not magic---clients still must poll periodically to ensure they don't miss any sensor measurements.

Just like `get_measured`, `collect_measured` might be empty when a daemon first starts. This trait should be totally orthogonal to has-measure-trigger.

## message: collect_measured

parameter: measurement_id: integer default: Null

response: {type: array, items: {type: array, items=[float, {type: map, values: [float, ndarray]}]}}

# Rejected Ideas

It might be more performant to transport this additional data by adding an additional dimension to each channel and making them all numpy arrays. The possible performance improvements would be outweighed by additional complexity for the daemons and clients.

# Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.