Saving BLE client service discovery on a PICO_W: How to create a uctypes struct layout that contains an array of another uctypes struct layout?

[brianreinhold]

First, why do this? The memory fragmentation on the PICO is so severe that the most robust way to avoid it is to predefine buffers of sufficient and fixed size to hold the data one needs. While the application is running, this buffer is used and reused as needed.

In this case I am writing a Bluetooth LE client that communicates with health devices. There is certain information about each health device that one needs to persist in order to reconnect without user involvement. This information is saved to a file for each device. A list of addresses for the device is independently maintained.

When a device is discovered, the list of addresses is checked, and if it is found, the saved data is read from the file and placed in the reserved uctypes struct.

How does one do this? First one defines a layout (mimicking a class C struct)

phd_info_layout = {
    "addr_type": (uctypes.UINT8 | 0),        # Type of address (random ,public) Needed for the lower levels in the delete bonding
    "pairable": (uctypes.UINT8 | 1),         # Whether or not the device is pairable. For proprietary devices this information might be known.
    "bonded": (uctypes.UINT8 | 2),           # Whether or not the device is bonded. THis will always be False during discovery.
    "encrypted": (uctypes.UINT8 | 3),        # Whether or not the device connection is encrypted. Set to False at the start of every connection
    "connected": (uctypes.UINT8 | 4),        # Whether or not the device is connected. Need this since the aioble device state is too slow to get that
    "regulated": (uctypes.UINT8 | 5),        # Whether or not the device is regulated from the reg cert data list of the DIS or from proprietary information
    "battery_level": (uctypes.UINT8 | 6),    # Battery level in percent (Typcially from Battery Service)
    
    "device_name_len": (uctypes.UINT8 | 7),  # Length fields for strings of variable length
    "model_len": (uctypes.UINT8 | 8),
    "manufacturer_len": (uctypes.UINT8 | 9),
    "serial_len": (uctypes.UINT8 | 10),
    "firmware_len": (uctypes.UINT8 | 11),
    "hardware_len": (uctypes.UINT8 | 12),
    "software_len": (uctypes.UINT8 | 13),
    "service_uuid_len": (uctypes.UINT8 | 14),
    "connection_good": (uctypes.UINT8 | 15),                # When true, Indicates that a connection concluded successfully
                                                            # This is used to NOT create a new entry (say on pairing) on a disconnect due to an error 
    
    "bt_addr": (uctypes.ARRAY | 16, uctypes.UINT8 | 12),      # Bluetooth Address as 12 ascii codes for 12-digit HEX string
                                                              # The original address passed up from aioble is a set of 6-bytes
                                                              # and the method 'hex()' or ubinascii.hexlify() unis called on it
                                                              # to convert it to the 12-bytes of ascii code b'aabbccddeeff'. 
    "device_name": (uctypes.ARRAY | 28, uctypes.UINT8 | 24),  # Device name (truncated to 24 characters)
    "system_id": (uctypes.ARRAY | 52, uctypes.UINT8 | 16),    # The EUI-64 system id as 16 ascii codes for the 16-digit HEX string
    "model": (uctypes.ARRAY | 68, uctypes.UINT8 | 16),        # Model number from DIS or equivalent
    "manufacturer": (uctypes.ARRAY | 84, uctypes.UINT8 | 24), # Manufacturer name from DIS or equivalent
    "serial": (uctypes.ARRAY | 108, uctypes.UINT8 | 16),      # Serial number from DIS or equivalent
    "firmware": (uctypes.ARRAY | 124, uctypes.UINT8 | 16),    # Firmware version from DIS or equivalent
    "hardware": (uctypes.ARRAY | 140, uctypes.UINT8 | 16),    # Hardware version from DIS or equivalent
    "software": (uctypes.ARRAY | 156, uctypes.UINT8 | 16),    # Software version from DIS or equivalent
    "service_uuid": (uctypes.ARRAY | 172, uctypes.UINT8 | 36),
    "specialization": (uctypes.UINT32 | 208),
    "last_connected": (uctypes.ARRAY | 212, uctypes.UINT8 | 25),      # Time last connected. FHIR string 2024-06-07T13:25:38+00:00
    "welch_allyn_password": (uctypes.UINT32 | 240),                      # Stores the password from Welch allyn device set up
    "welch_allyn_account_id": (uctypes.ARRAY | 244, uctypes.UINT8 | 8),  # Welch allyn account id which is 8 digits
}

Sorry for the big size but they are all basically strings, ints, and booleans. That aside, one then defines the buffer that is to hold this data and then one creates an 'instance' of the phd_info_layout struct.

phdinfo_size: int = 252 
phdinfoBuffer: bytearray = bytearray(phdinfo_size)
phdinfo_struct: uctypes.struct = uctypes.struct(uctypes.addressof(phdinfoBuffer), phd_info_layout)

The backing buffer is phdinfoBuffer. One can then use phdinfo_struct like a C struct, for example
phdinfo_struct.manufacturer = "A&D Medical".encode('utf-8')

This works fine. However, what I have NOT been able to figure out is how to add and element that is itself a layout.

Why do I want to do that? A serious shortcoming of the aioble library is that is does NOT save service discovery results and thus one must do service discovery every connection. A bonded peripheral does not expect this, and once encryption is established, is indicates or notifies any data it has, and the client is still engaged in service discovery, and once that is done, invokes the subscription. By then the measurement data is long gone and forever lost. Not good!!!

So I want to add to the phdinfo_struct saved service discovery results. One can define a layout for a service as follows:

service_layout = {
    "uuid": (uctypes.ARRAY | 0, uctypes.UINT8 | 36),            # Service uuid
    "uuid_len": (uctypes.UINT16 | 36),                          # 2 bytes or 36 bytes
    "start": (uctypes.UINT16 | 38),                             # start handle
    "end": (uctypes.UINT16 | 40),                               # end handle
    "chars": (uctypes.ARRAY | 42, characteristic_layout, _NUM_OF_CHAR)        # characteristics
}

The problem is the characteristics. A service can have several characteristics, in this case '_NUM_OF_CHAR' characteristics. I can define a layout for a single characteristic as follows:

characteristic_layout = {
    "uuid": (uctypes.ARRAY | 0, uctypes.UINT8 | 36),            # Characteristic uuid
    "uuid_len": (uctypes.UINT16 | 36),                          # 2 bytes or 36 bytes
    "value": (uctypes.UINT16 | 38),                             # value handle
    "end": (uctypes.UINT16 | 40),                               # end handle
    "properties": (uctypes.UINT16 | 42),                        # properties
    "cccd_handle": (uctypes.UINT16 | 44)                        # CCCD descriptor handle
}

However, what I do NOT know how to do (I have tried many things) is how to add an array of characteristic_layouts to my service_layout. If I ever figure out how to do that, then I can add an array of service_layouts to my phdinfo_layout.

I would appreciate any help on how to do this!!!