Does executorch run on esp32-s3?

[jaguilar]

Friends,

I've been working on a self-balancing robot project where I was going to use data from an actual robot to learn (e.g. using SAC via Stable-Baselines3 or similar). The general idea of the project was that each run of the collector, I'd ship a model to the bot, execute it and collect data, and stream by experiences to the client. The client would then train on the experiences, ship a new model to the bot, and repeat until I had a working self-balancing robot.

I initially started with stable-baselines3, but I couldn't find a good way of executing the pytorch model on my ESP32-S3. I know that in theory you can do pytorch->onnx->tflite micro, but in practice I was unable to get that working. onnx_tensorflow has been abandoned for three years, and I couldn't find any other straightforward means of accomplishing my goals. Eventually I gave up and switched over to tensorflow and tf-agents, but that has not been nearly as smooth of an experience as stable-baselines3 was.

Today, someone informed me about executorch. I'm given to understand executorch can run on baremetal. Does anyone have experience using it with an esp32-s3? Brief googling does not turn up any experience reports, but I thought asking here might get some insight from people who are familiar with the project. Particularly, I would like to know how difficult it is to get up and running. Getting tflite micro running on ESP-IDF was pretty cinchy -- it was actually the only part of the entire experience of using tensorflow that has been easy.

[digantdesai]

I don't think I have tried it but it should, the runtime and the portable library are quite portable. We regularly run on bare metal Cortex-M and Hifi DSPs, FWIW. Have to tried cross compiling and running it?

[mergennachin]

@jaguilar

ESP32-S3 is based on RISC-V processor, right?

Technically it should work. We have internally cross-compiled to RISC-V based toolchains (though we don't have a continous CI test to validate it).

also, as @digantdesai mentioned, we've also done work on baremetal cortex-M and cadence DSPs (https://github.com/pytorch/executorch/tree/main/backends/cadence)

if you get it working, we'd like to invite to you to contribute to our codebase