feat: Batch requests and extract fn/paths from addr2line (IDFGH-15433)

[nebkat]

Description

The current procedure of invoking addr2line individually for each address is rather inefficient, which causes large delays when there is a long backtrace printed in esp-idf-monitor.

This change instead groups addresses by their appropriate ELF file and requests all of them at once. The output is then parsed to map back to the specific input addresses.

In addition to this, the function, path and line are extracted to provide for better formatting in esp-idf-monitor.

Related

espressif/esp-idf-monitor#27

Testing

Tested locally with a variety of inputs.

---

Checklist

Before submitting a Pull Request, please ensure the following:

• 🚨 This PR does not introduce breaking changes.
• All CI checks (GH Actions) pass.
• Documentation is updated as needed.
• Tests are updated or added as necessary.
• Code is well-commented, especially in complex areas.
• Git history is clean — commits are squashed to the minimum necessary.

[nebkat]

Separate commit to show changes - can squash if necessary.

[Copilot]

Pull Request Overview

This PR improves the performance and output formatting of the panic decoder by batching addr2line requests and extracting function names, file paths, and line numbers from the tool output.

• Groups addresses by ELF file to reduce latency
• Parses addr2line output into structured trace entries, including inlined functions
• Replaces individual address decoding with batched processing

Comments suppressed due to low confidence (2)

esp_idf_panic_decoder/pc_address_decoder.py:105

• [nitpick] Since the parameter now expects a list of addresses, consider renaming 'pc_addr' to 'addresses' for improved clarity.

def lookup_pc_address(
        self,
        pc_addr: List[str],

esp_idf_panic_decoder/pc_address_decoder.py:116

• The previous addr2line command used the '-pfiaC' flags, including the '-p' flag for pretty printing. Verify if the omission of '-p' in the new command is intentional and ensure it does not affect the output format expected for parsing.

cmd = [f'{self.toolchain_prefix}addr2line', '-fiaC', '-e', elf_file, *pc_addr]

[peterdragun]

Separate commit to show changes - can squash if necessary.

Thank you for the update. Please squash commits.

The CI failure is not related to these changes, I will fix this ASAP. In the meantime, there are two issues reported by pylint that need attention:

pylint...................................................................Failed
- hook id: pylint
- exit code: 24

************* Module esp_idf_panic_decoder.pc_address_decoder
esp_idf_panic_decoder/pc_address_decoder.py:45:31: C0103: Argument name "t" doesn't conform to snake_case naming style (invalid-name)
esp_idf_panic_decoder/pc_address_decoder.py:46:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)

[dobairoland]

@nebkat Thank you for the contribution. It is a nice idea. I have left several recommendations in order that the code will be clearer for later maintenance.

[nebkat]

@dobairoland Have implemented your suggestions.

The text output remains completely unchanged (which is just like the original output of addr2line with pretty-print enabled):

Backtrace: 0x40376121:0x3fcb5590 0x40384ef9:0x3fcb55b0 0x4202c8c9:0x3fcb55d0 0x4202c8d3:0x3fcb55f0 0x4201101d:0x3fcb5610 0x40386499:0x3fcb5640 0x40386535:0x3fcb5660 0x403866b0:0x3fcb5690 0x40385a3e:0x3fcb56c0
--- 0x40376121: panic_abort at /IDF/components/esp_system/panic.c:454
--- 0x40384ef9: esp_system_abort at /IDF/components/esp_system/port/esp_system_chip.c:87
--- 0x4202c8c9: std::enable_if<is_invocable_r_v<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view>, void>::type std::__invoke_r<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view>(freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view&&) at /IDF_PROJECT/main/src/app/dispatcher.cpp:243
--- (inlined by) operator()<freertos::timer_view> at /IDF_PROJECT/troo-idf-cxx/freertos_cxx/include/freertos/timer.hpp:252
--- (inlined by) __invoke_impl<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()> >(char const*, app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()>, freertos::ticks, bool, bool)::<lambda(auto:97)>&, freertos::timer_view> at /TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/invoke.h:61
--- (inlined by) __invoke_r<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()> >(char const*, app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()>, freertos::ticks, bool, bool)::<lambda(auto:97)>&, freertos::timer_view> at /TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/invoke.h:111
--- 0x4202c8d3: void std::move_only_function<void (freertos::timer_view)>::_S_invoke<freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}>(std::_Mofunc_base*, freertos::timer_view&&) at /TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/mofunc_impl.h:219
--- 0x4201101d: std::move_only_function<void (freertos::timer_view)>::operator()(freertos::timer_view) at /TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/mofunc_impl.h:184
--- (inlined by) freertos::timer_owning::hook(tmrTimerControl*) at /IDF_PROJECT/troo-idf-cxx/freertos_cxx/include/freertos/timer.hpp:200
--- 0x40386499: prvProcessExpiredTimer at /IDF/components/freertos/FreeRTOS-Kernel/timers.c:

The changes are necessary to allow color formatting of the output:

In non pretty-print mode addr2line returns a more machine readable format:

0x40376121
panic_abort
/IDF/components/esp_system/panic.c:454
0x40384ef9
esp_system_abort
/IDF/components/esp_system/port/esp_system_chip.c:87
0x4202c8c9
std::enable_if<is_invocable_r_v<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view>, void>::type std::__invoke_r<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view>(freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}&, freertos::timer_view&&)
/IDF_PROJECT/main/src/app/dispatcher.cpp:243
operator()<freertos::timer_view>
/IDF_PROJECT/troo-idf-cxx/freertos_cxx/include/freertos/timer.hpp:252
__invoke_impl<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()> >(char const*, app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()>, freertos::ticks, bool, bool)::<lambda(auto:97)>&, freertos::timer_view>
/TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/invoke.h:61
__invoke_r<void, freertos::timer::timer<app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()> >(char const*, app::dispatcher::on_trigger_panic(const nlohmann::json_abi_v3_11_3::json&)::<lambda()>, freertos::ticks, bool, bool)::<lambda(auto:97)>&, freertos::timer_view>
/TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/invoke.h:111
0x4202c8d3
void std::move_only_function<void (freertos::timer_view)>::_S_invoke<freertos::timer::timer<app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}>(char const*, app::dispatcher::on_trigger_panic(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long, unsigned long long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&)::{lambda()#1}, freertos::ticks, bool, bool)::{lambda(auto:1)#1}>(std::_Mofunc_base*, freertos::timer_view&&)
/TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/mofunc_impl.h:219
0x4201101d
std::move_only_function<void (freertos::timer_view)>::operator()(freertos::timer_view)
/TOOLCHAIN/xtensa-esp-elf/include/c++/14.2.0/bits/mofunc_impl.h:184
freertos::timer_owning::hook(tmrTimerControl*)
/IDF_PROJECT/troo-idf-cxx/freertos_cxx/include/freertos/timer.hpp:200
0x40386499
prvProcessExpiredTimer
/IDF/components/freertos/FreeRTOS-Kernel/timers.c:653
0x40386535
prvProcessTimerOrBlockTask
/IDF/components/freertos/FreeRTOS-Kernel/timers.c:714
0x403866b0
prvTimerTask
/IDF/components/freertos/FreeRTOS-Kernel/timers.c:685
0x40385a3e
vPortTaskWrapper
/IDF/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:139

So most of the changes are for parsing this and recreating the pretty-print format.

One thing which could simplify understanding of the function output is if instead of returning a list with:

[..., ("0x4202c8c9", [location1, inline2, inline3]), ...]

It could return

[..., ("0x4202c8c9", location1), ("(inlined by)", inline2), ("(inlined by)", inline3), ...]

[dobairoland]

LGTM. Thank you for accepting my suggestions.

[peterdragun]

LGTM, thank you for updates. One for request: @nebkat Can you please rebase your MR? This should unblock the GH action for pre-commit.

[nebkat]

Suggested change

	if path == '??' and is_rom:
	path = 'ROM'
	if func == '??' and path == '??':
	continue
	if path == '??' and is_rom:
	path = 'ROM'

@peterdragun How about this? No point returning it from here either if there is no useful information.

[peterdragun]

That could work, but it would require some additional handling, as this results in the following output, which is not expected:

[nebkat]

Added valid variable to this loop - if any entry for a given address contains a function or path we keep the entire trace, otherwise if none of the entries do then we discard the address.

Probably overkill and not sure if this is even possible but this will ensure that if there was ever a situation like this we would keep the full trace, instead of suggesting that 0x40078000 == func at file.c:

0x40078000: ?? at ??:0
(inlined by) func at file.c:1
(inlined by) ?? at ??:0