Neural Control Mode

[SindreMHegre]

Solved Problem

Using neural networks for various tasks on UAVs in research is becoming more and more commonplace. My aim is to provide a module that can be used to replace the MC controller with NN to lower the barrier for using RL and NN on UAVs in general, and give researchers and hobbyist a place to start for NN use.

Solution

• Add TensorFlow Lite Micro as an inference lib into PX4
• Make custom board configs with neural controllers
• Make a module for NN control

Changelog Entry

For release notes:

Feature: Neural control mode for PX4, with possibility to use your own neural networks.
New parameters: MC_NN_EN, MIN_RPM, MAX_RPM, THRUST_COEFF
Documentation: read docs.px4.io/advanced/neural_networks, tflm and nn_module_utilities

Alternatives

We could also look into using Eigen or executorch

Test coverage

• Built and tested SITL with the x500
  https://review.px4.io/plot_app?log=52502d02-009e-4d1f-baac-a49dd51cfdef (check neural_control topic)
• Built and tested HITL with pixracerpro and jMAVSim Hitl, note that the sim itself seemed a bit unstable and the network is not optimized for this model.
  https://review.px4.io/plot_app?log=43d2ba6c-3d05-4d7f-a31b-490a57b3e647
• Tested live on ARL LMF model:
  https://review.px4.io/plot_app?log=d101189d-672e-42a3-9d39-828c675dde57
• Tested live on the X500, but some problems on the platform caused a sub optimal test.
  https://review.px4.io/plot_app?log=eff5ef61-3318-4f74-ad82-bfa42388dbd5

Video and paper on the module. Master thesis coming later
https://www.youtube.com/watch?v=lY1OKz_UOqM&t=1s
https://arxiv.org/abs/2505.00432

Context

Other feature requests:

• Put in different controller parameters with possibilities to replace different parts: Full e2e, only control, only allocation, Allocation and control as two separate networks.
• Script to give thrust and delay parameters from a flight log

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-sync-q-a-feb-19-2025/43827/2

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-sync-q-a-feb-19-2025/43827/1

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/how-to-integrate-eigen-library-with-px4-firmware/44065/2

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/adding-an-external-flight-mode-from-the-fcu/44137/1

[Pedro-Roque]

@Pedro-Roque I made some minor adjustments, mainly added a parameter to turn the possibility of manual control on and off. But manual control is possible now

Looks good to me! Feel free to close my PR on your fork.

I'll go once more through the structure tomorrow to ensure that there are no weird things with the rest of the mc_ integration. Thanks for all the effort @SindreMHegre !

[Pedro-Roque]

@SindreMHegre some comments remaining.

@MaEtUgR can you give a check to the current state here? This can now be used with RC. My major comment is that there is no attitude control as of now.

[SindreMHegre]

@Pedro-Roque The reward function gives very little reward for keeping the attitude, and attitude setpoint is not an input to the network, so I don't think it is feasible to add it. It's not meant to replace the classical controllers anyways, so I don't find it to be a big issue

[Pedro-Roque]

@Pedro-Roque The reward function gives very little reward for keeping the attitude, and attitude setpoint is not an input to the network, so I don't think it is feasible to add it. It's not meant to replace the classical controllers anyways, so I don't find it to be a big issue

Yes I agree that we shouldn't add non-tested work packages. I just wonder if the attitude will drift. I'd like to see a future version with this, if possible, though. But not a merge-stopper for me, unless @MaEtUgR thinks otherwise.

[SindreMHegre]

@dagar Had time to look at this yet?

[Jaeyoung-Lim]

Great work!

[SindreMHegre]

@mrpollo @Jaeyoung-Lim anything new happen today? Can this be merged now if @Pedro-Roque and/or @MaEtUgR approves?

[Pedro-Roque]

Approved on my side. @MaEtUgR should give final check as maintainer for multicopter :)

[Tfly6]

@SindreMHegre check SindreMHegre#4 . I couldn't build the code locally due to a tensorflow error, goes below:

[18/838] Building CXX object src/lib/tensorflow_lite_micro/CMakeFiles/tensorflow_lite_micro.dir/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc.o
FAILED: src/lib/tensorflow_lite_micro/CMakeFiles/tensorflow_lite_micro.dir/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc.o 
/usr/bin/ccache /usr/bin/c++ -DCONFIG_ARCH_BOARD_PX4_SITL -DENABLE_LOCKSTEP_SCHEDULER -DMODULE_NAME=\"tensorflow_lite_micro\" -D__CUSTOM_FILE_IO__ -D__PX4_LINUX -D__PX4_POSIX -D__STDC_FORMAT_MACROS -Dnoreturn_function="__attribute__((noreturn))" -I/home/roque/Gits/PX4-Autopilot/boards/px4/sitl/src -I/home/roque/Gits/PX4-Autopilot/platforms/posix/src/px4/common/include -I/home/roque/Gits/PX4-Autopilot/build/px4_sitl_neural -I/home/roque/Gits/PX4-Autopilot/build/px4_sitl_neural/src/lib -I/home/roque/Gits/PX4-Autopilot/platforms/posix/src/px4/generic/generic/include -I/home/roque/Gits/PX4-Autopilot/platforms/common -I/home/roque/Gits/PX4-Autopilot/platforms/common/include -I/home/roque/Gits/PX4-Autopilot/src -I/home/roque/Gits/PX4-Autopilot/src/include -I/home/roque/Gits/PX4-Autopilot/src/lib -I/home/roque/Gits/PX4-Autopilot/src/lib/matrix -I/home/roque/Gits/PX4-Autopilot/src/modules -I/home/roque/Gits/PX4-Autopilot/platforms/posix/include -I/home/roque/Gits/PX4-Autopilot/build/px4_sitl_neural/external/Install/include -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/flatbuffers/include -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/ruy -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/gemmlowp -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/cmsis/Cortex_DFP/Device/ARMCM7/Include -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/cmsis/CMSIS/Core/Include -I/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/tools/make/downloads/cmsis -O2 -g -DNDEBUG -std=gnu++17 -fPIC -g -fdata-sections -ffunction-sections -fomit-frame-pointer -fmerge-all-constants -fno-signed-zeros -fno-trapping-math -freciprocal-math -fno-math-errno -fno-strict-aliasing -fvisibility=hidden -include visibility.h -Wall -Wextra -Werror -Warray-bounds -Wdisabled-optimization -Wdouble-promotion -Wfatal-errors -Wfloat-equal -Wformat-security -Winit-self -Wlogical-op -Wpointer-arith -Wshadow -Wuninitialized -Wunknown-pragmas -Wunused-variable -Wno-missing-field-initializers -Wno-missing-include-dirs -Wno-unused-parameter -fdiagnostics-color=always -Wno-stringop-truncation -fno-builtin-printf -fno-strength-reduce -Wformat=1 -Wunused-but-set-variable -Wno-format-truncation -fcheck-new -Wreorder -Wno-overloaded-virtual -Wno-float-equal -Wno-shadow -MD -MT src/lib/tensorflow_lite_micro/CMakeFiles/tensorflow_lite_micro.dir/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc.o -MF src/lib/tensorflow_lite_micro/CMakeFiles/tensorflow_lite_micro.dir/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc.o.d -o src/lib/tensorflow_lite_micro/CMakeFiles/tensorflow_lite_micro.dir/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc.o -c /home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc
In file included from /home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/kernels/internal/reference/add_n.h:21,
                 from /home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/micro/kernels/add_n.cc:16:
/home/roque/Gits/PX4-Autopilot/src/lib/tensorflow_lite_micro/tflite_micro/tensorflow/lite/kernels/internal/common.h:33:10: fatal error: fixedpoint/fixedpoint.h: No such file or directory
   33 | #include "fixedpoint/fixedpoint.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
make: *** [Makefile:227: px4_sitl_neural] Error 1

EDIT: Fixed

Maybe you can try the command

make -f tensorflow/lite/micro/tools/make/Makefile run_keyword_benchmark

The comand will download some dependencies and test.

[SindreMHegre]

@Tfly6 thank you for the tip, I use the command:
tensorflow/lite/micro/tools/make/Makefile third_party_downloads
Which downloads the dependenices and runs some tests. So it does the trick

[Tfly6]

@SindreMHegre Your work is nice. I'm testing your code (for paper branch) in PX4 SITL Gazebo. How to change mode to Neural Control, Can you tell me the correct steps?

[SindreMHegre]

@Tfly6 Can we take this somewhere else, like mail?
sindre.hegre@gmail.com

[Tfly6]

@SindreMHegre OK! I had sent you email, but is your another email 😂
sindrheg@stud.ntnu.no

[hamishwillee]

Docs are "good enough".

[hamishwillee]

@MaEtUgR @dakejahl What is needed to get this through?

My only concern is the changes to files like src/drivers/gnss/septentrio/septentrio.cpp that seem unrelated.
Otherwise the design seems self-contained.

[Pedro-Roque]

@MaEtUgR @dakejahl What is needed to get this through?

My only concern is the changes to files like src/drivers/gnss/septentrio/septentrio.cpp that seem unrelated. Otherwise the design seems self-contained.

I think these should be fixed, they should not come from this PR. @SindreMHegre can you remove these changes? Or do you need them?

[SindreMHegre]

@hamishwillee @Pedro-Roque I agree that it does not necessarily fit it this PR, but the CI fails without the static casts, because when I include the standard libraries I get an error on ambiguous definitions. So this was the simplest solution I could come up with. But open to other ways to fix it:

PX4-Autopilot-public/src/drivers/gps/gps.cpp:452:37: error: call of overloaded 'abs(time_t)' is ambiguous
452 | int drift_time = abs(rtc_system_time.tv_sec - rtc_gps_time.tv_sec); ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.

[hamishwillee]

@Pedro-Roque I think this is the right places for the fix, but I don't know if it is the "right" fix - not a programmer.

[dakejahl]

The feature is self-contained and the changes to the build system are implemented correctly. I pulled out the GPS changes into a separate PR #25293. Once that PR is merged we can update this branch against main and get it merged. I see no reason to delay.

[Pedro-Roque]

Thanks @dakejahl ! Let's get this merged once all tests go through cleanly! :)

[github-actions]

/en/advanced/neural_networks.md

• LinkedFileMissingAnchor: #mc_nn_control not found in ../modules/modules_controller.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/modules/modules_controller.md)
• LinkedFileMissingAnchor: #MC_NN_THRST_COEF not found in ../advanced_config/parameter_reference.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/advanced_config/parameter_reference.md)

/en/advanced/nn_module_utilities.md

• LinkedFileMissingAnchor: #mc_nn_control not found in ../modules/modules_controller.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/modules/modules_controller.md)
• LinkedFileMissingAnchor: #mc_nn_control not found in ../modules/modules_controller.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/modules/modules_controller.md)
• LinkedFileMissingAnchor: #MC_NN_EN not found in ../advanced_config/parameter_reference.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/advanced_config/parameter_reference.md)
• LinkedInternalPageMissing: This linked file is missing: ../msg_docs/NeuralControl.md

/en/advanced/tflm.md

• LinkedFileMissingAnchor: #mc_nn_control not found in ../modules/modules_controller.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/modules/modules_controller.md)
• LinkedFileMissingAnchor: #mc_nn_control not found in ../modules/modules_controller.md (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/modules/modules_controller.md)
• LinkedInternalPageMissing: This linked file is missing: advanced/neural_networks.md

[dakejahl]

@hamishwillee is the above github-actions message about missing file anchors valid?

[SindreMHegre]

@dakejahl, I'm pretty sure he has looked them over before. I have checked at least and can't find any errors in the docs links

[mrpollo]

Here we go!!! Thanks, everyone, for the team effort. It's great to see this finally come in. I'm eager to see what people create with this.

[hamishwillee]

THanks for your patience @SindreMHegre !!! Very cool.