esp_player and A2DP (aadp) issue with sink mode, and source mode (AUD-4891)

[0x0fe]

ADF version 2.6
IDF version 5.0.2
Chip : ESP32D0WDR2 V3.1
IDE : VSCode / PIO

The device is an audio player providing 3 sources : sd_mmc, http and a2dp-sink and 2 outputs : i2s and a2dp-source.

I managed to get all these working with the low level APIs (pipelines and a ton of callback/handlers) but this is already very akward to use with a single source, and pretty unmanageable with multiple source / outputs.

I discovered the esp_player and esp_player_wrapper APIs, which make things a lot cleaner and easier to use,
so far i managed to get sd_mmc and http to i2s working fine, however i am facing troubles with a2dp.

So, in the player setup, i added a stream for a2dp, and tried my best to intilize the bluetooth service :

void *player_setup(void *cb, void *ctx){

    esp_audio_cfg_t cfg = DEFAULT_ESP_AUDIO_CONFIG();
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);

    cfg.vol_handle = board_handle->audio_hal;
    cfg.vol_set = (audio_volume_set)audio_hal_set_volume;
    cfg.vol_get = (audio_volume_get)audio_hal_get_volume;
    cfg.resample_rate = 48000;
    cfg.prefer_type = ESP_AUDIO_PREFER_MEM;
    cfg.cb_func = cb;
    cfg.cb_ctx = ctx;
    handle = esp_audio_create(&cfg);

    // Create readers and add to esp_audio
    fatfs_stream_cfg_t fs_reader = FATFS_STREAM_CFG_DEFAULT();
    fs_reader.type = AUDIO_STREAM_READER;

    // FatFS
    esp_audio_input_stream_add(handle, fatfs_stream_init(&fs_reader));

    // HTTP
    http_stream_cfg_t http_cfg = HTTP_STREAM_CFG_DEFAULT();
    http_cfg.event_handle = _http_stream_event_handle;
    http_cfg.type = AUDIO_STREAM_READER;
    http_cfg.enable_playlist_parser = true;
    audio_element_handle_t http_stream_reader = http_stream_init(&http_cfg);
    esp_audio_input_stream_add(handle, http_stream_reader);

    //http_stream_reader = http_stream_init(&http_cfg); // why twice???? i assume it is an error in the example
    //esp_audio_input_stream_add(handle, http_stream_reader); // why twice???? i assume it is an error in the example

    // A2DP SINK
    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA",
        .mode = BLUETOOTH_A2DP_SINK,
        .remote_name = "",
    };
    bluetooth_service_start(&bt_cfg);
    esp_audio_input_stream_add(handle,bluetooth_service_create_stream());
    //bt_periph = bluetooth_service_create_periph();//should we init manually?
    //esp_periph_start(set, bt_periph);//should we start manually?
    esp_bt_gap_set_scan_mode(ESP_BT_CONNECTABLE, ESP_BT_GENERAL_DISCOVERABLE);// shoud we set these manually?
    // where is AVRC play cmd issued? in the esp_player_music_play function? 

    // Add decoders and encoders to esp_audio
    mp3_decoder_cfg_t mp3_dec_cfg = DEFAULT_MP3_DECODER_CONFIG();
    mp3_dec_cfg.task_core = 1;
    esp_audio_codec_lib_add(handle, AUDIO_CODEC_TYPE_DECODER, mp3_decoder_init(&mp3_dec_cfg));

    // Create writers and add to esp_audio
    i2s_stream_cfg_t i2s_writer = I2S_STREAM_CFG_DEFAULT();
    i2s_writer.type = AUDIO_STREAM_WRITER;
    i2s_writer.i2s_config.sample_rate = 48000;
    i2s_writer.i2s_config.bits_per_sample = CODEC_ADC_BITS_PER_SAMPLE;
    i2s_writer.need_expand = (CODEC_ADC_BITS_PER_SAMPLE != I2S_BITS_PER_SAMPLE_16BIT);
    esp_audio_output_stream_add(handle, i2s_stream_init(&i2s_writer));

    // Set default volume
    esp_audio_vol_set(handle, 60);
    AUDIO_MEM_SHOW(TAG);
    ESP_LOGI(TAG, "esp_audio instance is:%p", handle);
    return handle;
}

Alas, while it does not crash the A2DP SINK is not discoverable, my mobile could not see it.
I tested the A2DP sink example, which uses the low level APIs instead of esp_player, and it works perfectly fine here.

I dont understand why the device is not discoverable, even though i explicitely call

esp_bt_gap_set_scan_mode(ESP_BT_CONNECTABLE, ESP_BT_GENERAL_DISCOVERABLE);

I also tried to initiate the playback, but it only leads to an error because there is no stream/peripheral

i also tried this, but no luck

    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA+",
        .mode = BLUETOOTH_A2DP_SINK,
        .remote_name = "",
        .user_callback.user_avrc_ct_cb = bt_app_avrc_ct_cb,
    };
    bluetooth_service_start(&bt_cfg);
    audio_element_handle_t bt_stream_reader = bluetooth_service_create_stream();
    esp_audio_input_stream_add(handle, bt_stream_reader);
    //esp_audio_input_stream_add(handle,bluetooth_service_create_stream());

    esp_periph_handle_t bt_periph = bluetooth_service_create_periph();
    esp_periph_start(set, bt_periph);
    vTaskDelay(100 / portTICK_PERIOD_MS);
    periph_bluetooth_discover(bt_periph);

To start the playback i use the following call, I found the very specific uri format in some header, needless to say it was not documented anywhere, i did not knew we were in linux...

esp_player_music_play("aadp://44100:2@bt/sink/stream.pcm", 0 /*info->offset*/, MEDIA_SRC_TYPE_MUSIC_A2DP);

E (40245) BLUETOOTH_SERVICE: Invalid Bluetooth periph, at line 793
I (40245) ESP_AUDIO_CTRL: Enter play procedure, src:0
I (40245) ESP_AUDIO_CTRL: Play procedure, URL is ok, src:0
I (40251) ESP_AUDIO_CTRL: resample_rate:48000, channels:2,rate:44100
I (40259) ESP_AUDIO_CTRL: Request_CMD_Queue CMD:0, Available:5, que:0x3ffe081c
I (40266) ESP_AUDIO_CTRL: Func:_ctrl_play, Line:776, MEM Total:2046844 Bytes, Inter:109063 Bytes, Dram:96323 Bytes
I (40277) ESP_AUDIO_TASK: It's a decoder
I (40282) ESP_AUDIO_TASK: 1.CUR IN:[IN_aadp],CODEC:[DEC_pcm],RESAMPLE:[48000],OUT:[OUT_iis],rate:44100,ch:2,pos:0
W (40292) ESP_AUDIO_TASK: No more in stream handle
E (40298) SMARTCONFIG: ESP_AUDIO_CALLBACK_FUNC, st:5,err:528385,src:0
I (40304) ESP_AUDIO_TASK: Func:media_ctrl_task, Line:830, MEM Total:2046844 Bytes, Inter:109063 Bytes, Dram:96323 Bytes
I (40305) ESP_AUDIO_CTRL: Exit play procedure, ret:81001

So, what am i doing wrong and how could i get the a2dp sink working?

Also linked is the problem of a2dp source, zero example or reference in the whole ADF( and even github). Unsure how we are supposed to set it up correctly. Below are my attemps :

void *player_setup(void *cb, void *ctx){

    esp_audio_cfg_t cfg = DEFAULT_ESP_AUDIO_CONFIG();
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);

    cfg.vol_handle = board_handle->audio_hal;
    cfg.vol_set = (audio_volume_set)audio_hal_set_volume;
    cfg.vol_get = (audio_volume_get)audio_hal_get_volume;
    cfg.resample_rate = 48000;
    cfg.prefer_type = ESP_AUDIO_PREFER_MEM;
    cfg.cb_func = cb;
    cfg.cb_ctx = ctx;
    handle = esp_audio_create(&cfg);

    // Create readers and add to esp_audio
    fatfs_stream_cfg_t fs_reader = FATFS_STREAM_CFG_DEFAULT();
    fs_reader.type = AUDIO_STREAM_READER;

    // FatFS
    esp_audio_input_stream_add(handle, fatfs_stream_init(&fs_reader));

    // HTTP
    http_stream_cfg_t http_cfg = HTTP_STREAM_CFG_DEFAULT();
    http_cfg.event_handle = _http_stream_event_handle;
    http_cfg.type = AUDIO_STREAM_READER;
    http_cfg.enable_playlist_parser = true;
    audio_element_handle_t http_stream_reader = http_stream_init(&http_cfg);
    esp_audio_input_stream_add(handle, http_stream_reader);

    //http_stream_reader = http_stream_init(&http_cfg); // why twice???? i assume it is an error in the example
    //esp_audio_input_stream_add(handle, http_stream_reader); // why twice???? i assume it is an error in the example

    // Add decoders and encoders to esp_audio
    mp3_decoder_cfg_t mp3_dec_cfg = DEFAULT_MP3_DECODER_CONFIG();
    mp3_dec_cfg.task_core = 1;
    esp_audio_codec_lib_add(handle, AUDIO_CODEC_TYPE_DECODER, mp3_decoder_init(&mp3_dec_cfg));

    // Create writers and add to esp_audio
    i2s_stream_cfg_t i2s_writer = I2S_STREAM_CFG_DEFAULT();
    i2s_writer.type = AUDIO_STREAM_WRITER;
    i2s_writer.i2s_config.sample_rate = 48000;
    i2s_writer.i2s_config.bits_per_sample = CODEC_ADC_BITS_PER_SAMPLE;
    i2s_writer.need_expand = (CODEC_ADC_BITS_PER_SAMPLE != I2S_BITS_PER_SAMPLE_16BIT);
    esp_audio_output_stream_add(handle, i2s_stream_init(&i2s_writer));

    A2DP SOURCE
    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA",
        .mode = BLUETOOTH_A2DP_SOURCE,
        .remote_name = "",
    };
    bluetooth_service_start(&bt_cfg);
    esp_audio_output_stream_add(handle,bluetooth_service_create_stream());

    // Set default volume
    esp_audio_vol_set(handle, 60);
    AUDIO_MEM_SHOW(TAG);
    ESP_LOGI(TAG, "esp_audio instance is:%p", handle);
    return handle;
}

With this configuration, i can see the ESP32 connects to the A2DP sink device, and the connection is correct (i get a notification on my sink device) however when i start the playback from http, it plays to the I2S instead of the a2sp sink device.
So after parsing the APIs i found the following and tried to set it up by guesswork (no reference anywhere) :

    esp_audio_setup_t sets;
    sets.set_codec="NAU88C22";
    sets.set_uri=play_test_url;
    sets.set_in_stream="HTTP_STREAM";
    sets.set_out_stream="A2DP_STREAM";
    esp_audio_setup(handle, &sets);

But it did not work, (play_test_url is an http link to a mp3 file, i did this exact configuration with the low level apis and it works, http stream to a2dp source, playing on the sink device.

As a side note : using "tags", especialy when they are not documented, is not appropriate. Where are these "tags" listed? is it the "debug" tag? are we supposed to guess them? It may be ok for linux, but it is quite unsuitable for embedded systems.

Anyway i think the a2dp source can work, as long as the esp_player can be changed to the correct output, at least the bluetooth service is doing its job, searching and connecting to the sink device.

Also i wonder how to set both sink and source instanciated in the esp_player setup, they are never used at the same time, of course, but i need them both registered so i can switch from one mode to the other at runtime.

are we supposed to init the bluetooth_service twice, once with source and once with sink (i dont think so)

    // A2DP SINK
    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA",
        .mode = BLUETOOTH_A2DP_SINK,
        .remote_name = "",
    };
    bluetooth_service_start(&bt_cfg);
    //A2DP SOURCE
    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA",
        .mode = BLUETOOTH_A2DP_SOURCE,
        .remote_name = "",
    };
    bluetooth_service_start(&bt_cfg);

are we supposed to merge the two enums? (i doubt)

    // A2DP SINK + SOURCE
    bluetooth_service_cfg_t bt_cfg = {
        .device_name = "FABA",
        .mode = BLUETOOTH_A2DP_SINK | BLUETOOTH_A2DP_SOURCE,
        .remote_name = "",
    };
    bluetooth_service_start(&bt_cfg);

Thanks in advance and clear example of simple function blocks are really lacking in ADF, it seems quite ncessary to provide clear examples with esp_player for each and every source for example, plus various other things such as OTA without tone partition, BLE PROV (which is not even implemented), etc. These are basic function everyone use and need with ADF.

[jason-mao]

@0x0fe I think you need the component as https://github.com/espressif/esp-adf/tree/master/examples/korvo_du1906/components/audio_player. One thing to note is that sink and source cannot be enabled at the same time.

[0x0fe]

@jason-mao Oh, i just noticed it yesterday and wondered wether it was better than the wrapper i use. Of course sink and source are only used exclusively, we will either use sink to I2S, or sd_mmc/http to source.
I am going to test it and yesterday i noticed the downmix is used inside, may i know if it allows us to play sone tone over the playback? If so i hope it may work here because i still have this problem with the downmix. #1069

[0x0fe]

@jason-mao
Sadly, the player component you pointed out does not work correctly, it is quite outdated, at first i spent quie some time to extract the usage from Korvo project, to find that it was not working correct, the bluetooth is not discoverable, A2DP related functions do not seem to work. Then i tried to run the unity tests and i found that many are failing, in particular the bluetooth doesnt work. And in fact the unity tests are still using APIs that do not exist anymore in IDF 5, so that means it has not ben fully tested with IDF 5 i guess. Somebasic functions work fine (sd_mmc, http) but at least A2DP is not working correctly, and tone API also has issues.