doc: start replacing is_mac_included

[IvoGruber]

This is just a starting point to get early feedback, replace PR #323 and continue discussion on issue #42.

Not implemented yet and not ready to merge.

This would be my first second proposal to solve use case described in issue #42:

1. Generally do not use MAC address of the device in the nonce.
2. Instead put it as the first 6 bytes of the unencrypted payload.
3. Encrypt this payload if requested by associated flag.
4. Thus the MAC is hidden in encrypted mode.
5. When decrypting, MAC address is not needed anymore (in nonce).
6. After decryption the MAC address is known now.

We can burn another bit in the adv_info to support that mode (is_mac_included_privately).

To decide later:
With V3 of BTHome we can make that obligatory and can free that bit again, but this needs always 6 extra bytes?!
Thus we will have the MAC address in encrypted and unencrypted mode. In unencrypted mode and environments that hide/randomize the MAC like MacOS we can still map the advertisement to the correct device. In encrypted mode we automatically will have privacy if the environment supports that by hiding/randomizing the MAC address. Encryption will be recommended if privacy is needed.

[codecov]

Codecov Report

❌ Patch coverage is 53.33333% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.54%. Comparing base (1c8d3a1) to head (6346f23).

Files with missing lines	Patch %	Lines
src/bthome_ble/parser.py	53.33%	4 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #328      +/-   ##
==========================================
- Coverage   83.21%   82.54%   -0.67%     
==========================================
  Files           6        6              
  Lines         685      699      +14     
  Branches      101      105       +4     
==========================================
+ Hits          570      577       +7     
- Misses         88       92       +4     
- Partials       27       30       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Ernst79]

Suggested change

	# It was implemented to adsress issue #42
	# It was implemented to address issue #42

[matt7aylor]

Hi guys, thanks for keeping this in the BTHome plans and keeping us informed.

Reason for wanting MAC address in packet

As far as I understand it all implementations of BTHome use the physical MAC address to identify a device (e.g. in Home Assistant the MAC address is directly visible in the entity and required to monitor for data). The MAC address can generally be obtained directly from the OS/radio header (on linux/android/...). Thus in most circumstances it is not necessary for it to be included in the packet payload directly.

The complication is if you want to identify the sensor on an Apple product (e.g. a phone app with BTHome parsing) and be able to know which sensor you are looking at. Apple do not let you see the MAC directly and instead give you a random id. So for the subset of people who want to be able to identify their sensors on multiple platforms it is necessary to put the MAC address (or other unique identifier) into the packet payload. This practice is quite common among BLE sensor manufacturers, particularly where they might have a phone app to scan for and view the sensor data. The MAC address of the sensor is generally included in the manufacturers packet payload and the MAC address is generally printed on the device itself as text or QR code, making it much easier to identify the sensors. (Imagine having 10 temperature sensors and you want to identify which data is from which sensor with your iPhone.)

In the original issue we did not touch upon encryption, for our own use cases we have not needed/used it. The original issue was purely about being able to identify a sensor as the same sensor e.g. scanning with an iPhone app and home assistant.

Proposed Solution

1. Generally do not use MAC address of the device in the nonce.

Good point. If the goal is a packet that can be decrypted on iPhone/Mac then needing the MAC address to create the nonce to decrypt the packet, when the only way to get the MAC is from the encrypted payload, is a bit of a non-starter. So the MAC either needs to be plain text or the decryption process needs to not depend upon it.

2. Instead put it as the first 6 bytes of the unencrypted payload.

Is this intended to be optional? There are situations where including the MAC in the payload is necessary but I can see that for many implementations being able to skip those 6 bytes could be very useful for including more data in one packet and extending battery life. For example, if a user just wanted to have a sensor on home assistant then including the MAC in the payload is redundant. Using the same bit 1: “MAC included” flag would seem ideal in this case.

3. Encrypt this payload if requested by associated flag.

The existing bit 0: “Encryption flag”?

4. Thus the MAC is hidden in encrypted mode.
5. When decrypting, MAC address is not needed anymore.
6. After decryption the MAC address is known now.

This makes sense. I would argue that Apple are being overly sensitive about the visibility of the MAC address and that for this class of devices there isn't really a privacy benefit, especially if you can disable the inclusion of the MAC in the packet (maybe for personal sensors carried around by an iPhone user who was worried about having their location tracked by a group of committed Apple users?). But if the MAC can be included in the encrypted payload without losing any cryptographic functionality then that could make sense as it avoids any potential issue.

We can burn another bit in the adv_info to support that mode.

The bit 1: “MAC included” flag from #42 ?

With V3 of BTHome we make that obligatory and can free that bit again.

As mentioned above, the packet space is tight and many users may prefer to save those 6 bytes.

I haven't looked at the encryption side at all but potentially the MAC address is in the nonce for a good reason? Possibly it was useful to pad out the nonce somewhat for a bit more cryptographic security? Or perhaps with a plain text MAC you could implement a lookup for different encryption keys based on the MAC address, which might be more valuable than any potential privacy benefit?

Perhaps for v2 it would be simpler to have the option to include the MAC as plaintext using the existing undocumented flag (enabling using the existing encryption/decryption method) and maybe in a v3 to consider encrypting the MAC address in the payload (and any cryptographic/other implications)?

[IvoGruber]

I haven't looked at the encryption side at all but potentially the MAC address is in the nonce for a good reason?

I am not an expert, but I would argue, that this is only good for users that use same encryption key on different devices. In those cases an attacker would be able to replay a signal with higher encryption counter on the device with lower encryption counter since encrypted messages do not differ on those devices. Alternatively you can use some random bytes to prevent that also, but those also have to be transmitted. Another way to prevent this attack is to put this in the payload, which is suggested here.

[IvoGruber]

As mentioned above, the packet space is tight and many users may prefer to save those 6 bytes.

Thx for this advice. I will check how to adjust the proposal to be more flexible on that.

[IvoGruber]

Added a first implementation attempt. The encrypted MAC is not usable yet. Probably needs more refactoring first. Tests are missing.