Live monitoring of the minknow using the API

[jjkoehorst]

I am working on a remote monitoring / storage system and I would like to connect the gridion we have running in our lab.
Using the API I am able to access a simulation run that is running on my laptop and I am able to retrieve basic information like which flow cells are connected and so forth

        print(f"""Acquired: {acquired}, Processed: {processed} for flow cell '{position}' host '{position.host}' state '{position.state}' device_type '{position.device_type}' 
              parent_name '{position.parent_name}' description '{position.description}' 
              protocol_state '{position.protocol_state}' is_simulated '{position.is_simulated}'""")

Acquired: 6978944, Processed: 6978496 for flow cell 'MS00000 (running)' host 'localhost' state 'running' device_type 'MINION' 
              parent_name 'MS00000' description 'name: "MS00000"

However when trying to go beyond that for example get current temperature connection.statistics.get_temperature() it just hangs.

Also I am not sure yet how to retrieve the other information like available pores, reads, bases, health etc...

I tried going through the python tests but these functions might not be used in those?

[0x55555555]

Hi @jjkoehorst ,

Can you attach a demonstration script you are running which hangs?

Do the examples in this project run for you?

The API calls in the statistics file will generally return streams of information which you may have to handle differently to standard unary calls, but without an example I can't be sure thats whats causing issues.

Thanks,

• George

[jjkoehorst]

Ok found out that hanging was my fault I did a parallel processing for each of the different flow cells but did not realise it would not throw visible errors. I have the complete acquisition info at the moment.

Channel Callback: ChannelConnectivity.READY
Traceback (most recent call last):
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/gridion.py", line 242, in <module>
    main(manager)
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/gridion.py", line 113, in main
    process_flow_cell(pos)
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/gridion.py", line 102, in process_flow_cell
    minion_flow_cell(position)
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/gridion.py", line 96, in minion_flow_cell
    print(f"bla : {connection.statistics.stream_temperature(run_id)}")
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/venv/lib/python3.9/site-packages/minknow_api/statistics_service.py", line 370, in stream_temperature
    return run_with_retry(self._stub.stream_temperature,
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/venv/lib/python3.9/site-packages/minknow_api/statistics_service.py", line 78, in run_with_retry
    result = MessageWrapper(method(message, timeout=timeout), unwraps=unwraps)
  File "/Users/koeho006/git/m-unlock/lab_equipment/bdp/gridion/minknow_api_implementation/venv/lib/python3.9/site-packages/grpc/_channel.py", line 1392, in __call__
    raise rendezvous  # pylint: disable-msg=raising-bad-type
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "Exception serializing request!"
        debug_error_string = "None"
        

Might be due to the fact I am running a simulation from https://github.com/looselab/readfish?tab=readme-ov-file#adding-a-simulated-position-for-testing which might not have this information as it is not the gridion we have in the lab.

[jjkoehorst]

It looks like that acquisition_info = str(connection.acquisition.get_acquisition_info()) contains most of the live stream information

[0x55555555]

That will contain a lot of summary information from the run yes.

If you need assistance with the stream_temperature call above, can you post code showing how you are invoking it?

From the traceback, it might be:

connection.statistics.stream_temperature(run_id)

Should be:

connection.statistics.stream_temperature(acquisition_run_id=run_id)

Thanks,

• George

[jjkoehorst]

Now we are getting a step further indeed

print(f"Second attempt : {connection.statistics.stream_temperature(acquisition_run_id=run_id)}")
Second attempt : <_MultiThreadedRendezvous object>

[jjkoehorst]

and looping over that returns many

temperatures {
  minion {
    asic_temperature: 32.976749420166016
    heatsink_temperature: 35.739429473876953
  }
  target_temperature {
    minimum: 30
    maximum: 30
  }
}

[jjkoehorst]

        print(f"Second attempt : {connection.statistics.stream_temperature(acquisition_run_id=run_id)}")
        x = connection.statistics.stream_temperature(acquisition_run_id=run_id)
        for i in x:
            print(i)

I do notice that when its printing it never continues but it could be that this should be executed in a separate thread?

[0x55555555]

It will continue to stream back temperatures from the device every minute until you cancel the stream.

You could specify a start and/or end time to the API if you don't want an indefinite stream, but it will depend on what you are trying to achieve.

• George

[jjkoehorst]

I am trying to pull the current temperature once a minute while its running

[jjkoehorst]

        x = connection.statistics.stream_temperature(acquisition_run_id=run_id, _timeout=1)
        print(f"bla: {dir(connection.statistics)}")
        for index, i in enumerate(x):
            message = MessageToJson(i)
            print(f"Index: {index}, Message: {message}")
            break
        print("DONE!!!")

but there is no "time" information associated with the values so I guess this is for a plot only?

      "targetTemperature": {
        "minimum": 30.0,
        "maximum": 30.0
      }
    },
    {
      "minion": {
        "asicTemperature": 32.620052337646484,
        "heatsinkTemperature": 35.70883560180664
      },
      "targetTemperature": {
        "minimum": 30.0,
        "maximum": 30.0
      }
    },
    {
      "minion": {
        "asicTemperature": 32.61387252807617,
        "heatsinkTemperature": 35.69221115112305
      },
      "targetTemperature": {
        "minimum": 30.0,
        "maximum": 30.0
      }
    },
    {
      "minion": {
        "asicTemperature": 32.656681060791016,
        "heatsinkTemperature": 35.70378112792969
      },
      "targetTemperature": {
        "minimum": 30.0,
        "maximum": 30.0
      }
    }
  ]
}```

[0x55555555]

Hello,

To calculate the absolute time of each bucket:

• Take the step value you provide the request (or default 60s)
• Work from the start time of of the acquisition period
• Adding step value for each bucket you receive

Thanks,

• George