Reconnection and pairing loss issue in Controller-only HCI mode (ESP32 - NINA-W106 4-Mb FLASH) (IDFGH-16822)

[tcpipchip]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Hi everyone,

I’m using a u-blox NINA-W106 (ESP32) module in Controller-only HCI mode over UART (H4), connected to a Raspberry Pi running BlueZ.
The use case involves Bluetooth Classic (BR/EDR) only, specifically SPP/RFCOMM communication with an Android application.

The ESP32 firmware is based on the controller_hci_uart_demo example, with the following adjustments:

CTS/RTS flow control enabled

Fixed baudrate (921600)

No sleep or power management enabled

Fixed BD_ADDR read from eFuse (not modified)

The HCI-UART transport is stable — there are no more “device down” or transport errors.
However, we are still facing a persistent reconnection and pairing issue.

🔹 Scenario and symptoms

The initial pairing between the Android tablet and the Raspberry Pi (via NINA) works correctly.

The SPP connection and data transfer operate flawlessly.

When the Android application is closed and reopened, it attempts to reconnect automatically.

At this point, btmon shows:

Link Key Request → followed by Link Key Request Negative Reply (host has no key);

Android initiates re-pairing;

Host responds with IO Capability Request Negative Reply (Pairing Not Allowed 0x18);

Result: Simple Pairing Complete – Authentication Failure (0x05).

Android then automatically removes the device from its paired devices list.

Important: the ESP32 was not reset during this test — only the Android app was closed and reopened.
Even so, the host (BlueZ) seems to lose the link key, causing reconnection failure.

🔹 What has been verified

Correct baudrate and hardware flow control (921600 + CTS/RTS).

No use of esp_bt_controller_disable() or controller resets between sessions.

BD_ADDR is fixed (read from eFuse).

hci0 interface remains UP and active throughout the test.

The problem does not occur when using CSR/WT11-based dongles (same host, same Android app).

🔹 Possible cause (based on ESP-IDF issue #12650)

“Multiple users in ESP32 community (btstack issue #44, ESP-IDF issue #1275, Google Groups thread) report that link keys are not stored/persisted in ESP32 context, which aligns with our scenario of Controller-only HCI mode.”

From what we’ve analyzed, it seems that the Controller-only HCI mode on ESP32 does not retain or persist BR/EDR link keys after pairing, nor keeps them valid between sessions.
This causes the host to request a missing link key, leading to the “Authentication Failure (0x05)” upon reconnection.

We would like to confirm with the Espressif team:

Is there currently any official support for BR/EDR link key persistence in HCI Controller-only mode?

If not, is there any API or vendor command that allows intercepting or restoring link keys from the controller side?

Are there plans for a future fix (for example, automatic NVS storage of link keys)?

Is there a supported method to intercept Link Key Notification / Link Key Request HCI events from the controller in ESP-IDF?

🔹 Project context

The NINA-W106 module is used as part of a custom Bluetooth dongle in an industrial production environment (integration with Linux-based self-service terminals).
Modifications to the Raspberry Pi or BlueZ host software are not possible, so the solution must be implemented entirely on the ESP32 firmware side.

Any guidance or clarification on BR/EDR link key persistence in controller-only HCI mode would be greatly appreciated.

Thank you very much,

[boblane1]

Hi @tcpipchip,

Thanks for reporting.

I have tested with Ubuntu and Android phone, I can not reproduce the issue. I found the Bluez host stack will save the link key. Here is my information:

Would you mind providing your test APP and HCI logs?

Is there currently any official support for BR/EDR link key persistence in HCI Controller-only mode?

ESP32 doesn't support link key persistence in HCI Controller-only mode, but we may support this feature in the future. For now, it only supports save link key by HCI command HCI_Write_Stored_Link_Key when power is not off.

Is there a supported method to intercept Link Key Notification / Link Key Request HCI events from the controller in ESP-IDF?

You can try with vhci mode.

[caiocasadio82]

Hi, thanks for the feedback on this issue.

I’m the user who originally reported the problem on Espressif’s forum, and since we didn’t receive a reply there, our hardware engineer opened this GitHub issue on my behalf.

As requested, I’m attaching the complete btmon log captured during a clean test of the HCI-UART controller running on an ESP32 module (u-blox NINA-W106).

Below is an explanation of the exact procedure used to generate the log, so you can clearly see the sequence of events:

Test Procedure Used to Generate the Log

I powered up a Raspberry Pi Zero 2 W running Linux (Raspberry Pi OS) and connected via SSH.

Before any Bluetooth activity, I started capturing a raw log using:

sudo btmon --write nina_hci.log

I attached the ESP32/NINA module as an HCI-UART controller using hciattach, and the controller initialized normally.

When the application on the ESP32 finished booting, the device became discoverable and ready to accept incoming pairing (this moment is not visible in the log, because HCI-UART mode does not output application logs).

On an Android tablet running our application, the app started scanning. After ~30–40 seconds it attempted to connect and the system displayed a pairing dialog.

I confirmed the pairing request on the tablet, and the first connection was successfully established.

Our application then performed several data exchanges (password requests in JSON format — they are visible in the HCI log).

After this, I closed the application on the tablet. At this point, the ESP32 was still listed as a paired/trusted Bluetooth device on the tablet.

When reopening the app, it attempted to reconnect. Multiple retries occurred, and each time the ESP32 (acting as HCI controller only) failed to provide a valid BR/EDR Link Key to BlueZ.

After several failed authentication attempts, the tablet automatically removed the ESP32 from its paired devices list.

From this moment on, the device could not reconnect anymore unless I manually cleared the bonding information on the Raspberry Pi host (via btmgmt clear).
This aligns with the expected failure mode when the Bluetooth controller does not persist or return BR/EDR link keys.

Observed Behavior (matches the reported issue)

First pairing works normally.

Upon reconnection, BlueZ sends Link Key Request.

The ESP32 controller responds with Negative Reply.

Authentication fails.

The remote device eventually unpairs itself automatically.

A new connection is only possible after manual bonding reset on the host.

This strongly suggests that the ESP32 acting as an HCI controller is not persisting or returning BR/EDR link keys, exactly as described in the initial report.

Attached Files

nina_hci.log — Raw btmon capture

nina_hci.txt — Decoded btmon output for easier review

Please let me know if you need:

a second log with more verbose settings,

a full synchronized capture including hcidump,

or a test using a different ESP-IDF branch or configuration.

Thanks again for taking the time to look into this.
We appreciate any insight about whether HCI-only + BR/EDR Link Key persistence is supported or expected to work on ESP32.

nina_hci.txt

[boblane1]

Hi @caiocasadio82,

> HCI Event: Link Key Notification (0x18) plen 23         #332 [hci0] 76.475284
        Address: 74:BE:F3:55:EC:19 (Motorola Mobility LLC, a Lenovo Company)
        Link key[16]: d81d15cc99f90b39411974fef964993f
        Key type: Authenticated Combination key from P-256 (0x08)
@ MGMT Event: New Link Key (0x0009) plen 26           {0x0001} [hci0] 76.476320
        Store hint: Yes (0x01)
        BR/EDR Address: 74:BE:F3:55:EC:19 (Motorola Mobility LLC, a Lenovo Company)
        Key type: Authenticated Combination key from P-256 (0x08)
        Link key[16]: d81d15cc99f90b39411974fef964993f
        PIN length: 0

The log shows that the Bluez has stored the link key at the beginning.

Bluetooth monitor ver 5.75
= Note: Linux version 6.6.74+rpt-rpi-v8 (aarch64)                      0.794065
= Note: Bluetooth subsystem version 2.22                               0.794106
= New Index: 00:00:00:00:00:00 (Primary,UART,hci0)              [hci0] 3.134946
= Open Index: 00:00:00:00:00:00                                 [hci0] 3.135079

I notice that the Bluetooth MAC address is not parsed correctly. This may be the reason why the Bluez has stored the link key but reply with HCI_Link_Key_Request_Negative_Reply. You can try to solve this issue first and check if the original issue still exists.

[caiocasadio82]

Hello,
thank you for your previous feedback.

First of all, I would like to apologize — the previous log I uploaded was incorrect.
During my tests I discovered that the “missing MAC” problem was caused by my own reconnect routine, because I was not re-attaching the UART to the HCI correctly.
This part is now fixed, so please disregard the earlier log.
Sorry for the confusion.

Now I am submitting three new logs, captured correctly and following a strict methodology:

first_connection.log

second_connection.log

third_connection.log

I converted all of them using btmon --read and verified that everything is consistent.

🔧 Test procedure (exact steps)

For each test, I did the following:

Before starting

I removed Bluetooth pairing on both sides (Raspberry Pi + Android tablet).

I powered everything off.

After booting the Raspberry Pi and connecting via SSH, I started logging using:

sudo btmon --write first_connection.log

🟢 FIRST CONNECTION (expected behavior)

ESP32 application booted and waited for incoming connection.

The Android tablet requested pairing normally.

I accepted the pairing and the link was established.

I exchanged several JSON messages through RFCOMM without problems.

Then I closed the Android app, which disconnected cleanly.

End of first log.

I powered everything off again.

This first connection behaves exactly as expected.

🟡 SECOND CONNECTION (problem)

On the second boot, the Android tablet shows the Raspberry Pi as already paired.

However, when I try to connect, the tablet asks for pairing again, even though the device is shown as paired.

I accept pairing again, and the link works normally.

I exchanged some JSON messages again, disconnected, and powered everything off.

🔴 THIRD CONNECTION (same problem again)

Exact same behavior as the second test:
Even though the Raspberry Pi is already shown as paired on the Android tablet,
Android asks for pairing again before it will connect.

❗ MAIN ISSUE

The core problem is:

The ESP32 (in HCI UART mode) does not accept or return the stored link key, causing Android to request pairing again at every new connection.

This is extremely problematic for our product because we depend on a stable bonding relationship.

In our real-world deployment, the user cannot pair again every time there is a disconnection or reboot.
This makes the solution unusable in production.

🔧 Comparison with WT11 (legacy hardware)

We have a legacy product based on the Bluegiga WT11.
Using WT11:

Pairing occurs only once.

The link key is stored internally.

Reconnects are automatic.

The Android tablet never asks to pair again.

We are now migrating to the u-blox NINA (ESP32) because WT11 is end-of-life.
However, this issue with link key persistence is blocking the migration.

❓ QUESTION

What is required on the ESP32 side (in BT HCI mode) so that:

the link key is stored,

the link key is returned on Link Key Request,

and Android does NOT ask for pairing again after each reboot?

Is there any configuration, HCI event handling or host-side requirement that we are missing?

We would appreciate guidance on how ESP-IDF/ESP32 expects link key persistence to work in HCI Controller mode, or if this is a known limitation.

I am attaching the three logs that demonstrate the issue clearly:

first_connection.log

second_connection.log

third_connection.log

Thank you very much.

first_connection.txt
second_connection.txt
third_connection.txt

first_connection.txt
second_connection.txt
third_connection.txt