Questions Regarding [Policy Deployment, Trigger Behavior, Documentation, and Logging]

[coolermzb3]

Hello, @jonas-eschmann !

Thank you so much for your paper and project! I’m deeply interested and have reproduced the entire work from scratch (starting with purchasing a Crazyflie) —— it’s really amazing!

However, I’ve run into a few issues and would greatly appreciate your guidance:

1. [Critical] Sudden Power Loss.
   When switching to my trained policy (e.g., with the default seed=0) by pressing button ("hover" on the gamepad, or "enter" in trigger.py), the drone frequently experiences a sudden power loss approximately 2 seconds after the motor starts, which cannot be powered on via the power button, unless the charger is plugged in.
   Switching to another actor.h (e.g., the pre-trained one in your repository or my seed=6 model) may reduce the issue. However, adjusting --height in trigger.py (e.g., to 0.2 or 0.3) always reproduces the issue. Is this the expected behavior, or am I making some mistakes?

2. send_hover_packet in trigger.py Seems Not Working?
   During takeoff_and_switch, the drone doesn't act as described. Instead, it waits on the ground for 3 seconds and then directly activates the policy. Am I missing something?

3. Documentation Updates
   3.1 The Deploy section only describes the Docker method without mentioning the Native one. Besides, the pre-built Docker image on DockerHub uses an older controller version (0a7680d), which lacks the motor_warmup feature (re-enabled in d52b11a), causing conflicts with the current trigger.py. To work around this, I reverse-engineered a native compilation script via Dockerfile_build_firmware, but it would be helpful to officially document this for other users. (I'd be happy to submit a PR, but perhaps it would be better for you to handle it to ensure accuracy?)
   3.2 The Gamepad section mentions modifying parameters like rlt.wn to 4, but I couldn’t find where to make these changes (in the cfclient? Or somewhere else?). Is it only for trigger.py?

4. Accessing LOGs/DEBUG_PRINTs
   I noticed logging statements (e.g., LOG and DEBUG_PRINT) in rl_tools_controller.c. Could you please tell me how to view these logs?

Any clarification would be greatly appreciated! Thank you again for your time and excellent work!

[jonas-eschmann]

Hi @coolermzb3 thanks for reaching out and sorry for the late reply!

1: I've experienced this kind of power loss sometimes as well but for me it was usually immediately after turning on the motors (taking off using the policy) usually not when activating the policy in mid-flight. It is indeed very weird that the battery or some other safety circuit seems to shut off and that it requires a disconnection of the battery to reboot. My fix was to add a warmup time before taking off with the policy as in the following line:

https://github.com/arplaboratory/learning_to_fly_controller/blob/d52b11a03caa1dadaf6b6bf02f328719b5bdd215/rl_tools_controller.c#L367

I believe, ramping the motors too fast causes a voltage drop. This might also be the reason why there is a motor ramp option in the cfclients UI

2: Does the hovering work when you use the cfclient UI and a controller (using the hover button)?

3.1: Makes sense with the Docker version, do you think it would be better to remove the Docker instructions from the readme and focus on native training (Ubuntu 24.04)? Why did you need to reverse engineer, did the Native Installation instructions not work?

In general, the goal is (as mentioned in the top note of the README.md) to maintain the L2F simulator as part of RLtools and keep the embedded controllers in sync with the monorepo there. We are currently working on a new release which hopefully comes out in a couple of weeks to clean everything up and bring it into lock step.

3.2 & 4: I'm mainly using a hacked version of the cfclient instead of the trigger.py but it is specific to our mocap setup and badly hacked together so I was hesitant to publish it 😅. But since multiple people asked for it, here it is rl-tools/crazyflie-clients-python. It is really annoying how cfclient handles gamepad mappings etc. so I can't really support with the setup but maybe it helps as an inspiration. When you use cfclient instead of the trigger.py you can see the console tab with the log outputs and you can also set parameters like rlt.wn (which you can also set in the trigger script btw.

Sorry for the late response again, I hope it helps a bit. Please let me know how it goes!

[coolermzb3]

Hey! Thanks for your enthusiastic reply and guidance. Sorry for my late response - hope you don't mind!

I realize I might not have been clear enough before, but now that I'm more familiar with the project, here are my new findings:

1&2 About power loss

Yes, I meant the motor loss during startup. It still happens occasionally, maybe related to battery level, even when activated by hover button in cfclient, which means the code warmup maybe fail. But I found a workaround - doing a manual warmup (push some throttle first then press the hover button, using cfclient and gamepad) helps a lot. After that, hover works perfectly!

3.1 About deploy

During installation/training, I had no problems (though I only tried Native on Ubuntu 20.04, not Docker or other versions - but I believe they should work fine). My issues were only during deployment, specifically when compiling the firmware. The README lacks native instructions here. The Docker version's controller also lacks the motor_warmup option. "Reverse-engineer" might have been the wrong word - I basically just followed the Dockerfile_build_firmware steps to get native compilation working:

Native build commands

# === only need to run once ===
sudo apt install gcc-arm-none-eabi
# in the original directory `learning_to_fly` 
git submodule update --init -- controller  
cd controller
git submodule update --init --recursive -- external/crazyflie_firmware
git submodule update --init -- external/rl_tools   # non-recursive because we only need the headers
cd external/crazyflie_firmware
make cf2_defconfig

# === need to run every times when building ===
# train, and pick an `actor_xxx.h`, put it into controller/data/actor.h, maybe do some backup, though
# from the original directory `learning_to_fly` 
mv controller/data/actor.h controller/data/actor.h.backup
cp checkpoints/xxxxxxx/actor_000000000300000.h controller/data/actor.h
mkdir -p build_firmware
cd controller
make -j$(nproc)
cp build/cf2.bin ../build_firmware/cf2.bin

Then one can continue to flash following the readme.

Though the controller commit version are slightly different(0a7680d in docker, d52b11a in submodule), both work equally well.

3.2&4. Thank you for your custom rl-tools/crazyflie-clients-python!

At first I couldn't find where to view rlt.wn and logs using your cfclient, then realized I was missing some menu options (orz). After enabling parameters/console under View-Tabs/Toolboxes, everything worked. Sorry, I'm such a noob! Even the latest cfclient works this way, but I prefer your version because (after much trial, config, and code reading) I found it can respond to right-button messages (when trigger=0) to switch between learned and basic PID modes(left, originally hover button) mid-air. Though there's a small bug: switching from learned to PID is seamless, but PID→learned always causes a brief motor stop(no matter motor_warmup is True or False) with ~1m height drop (didn't dare test higher). Is this expected behavior?

About trigger.py - I think the issue is send_hover_packet not working (can't verify since cfclient can't connect simultaneously), but no big deal since send_learned_policy_packet works. Besides, now I prefer using cfclient anyway.

One new annoying issue: sometimes cfclient(yours and offcial both) fails to connect, multiple times, especially after power loss. The console shows:

SYS: The system resumed after a failed assert [WARNING]
SYS: Assert failed at ~/project/learning_to_fly/controller/external/crazyflie_firmware/src/hal/src/radiolink.c:164
SYS: cfAssertNormalStart [FAIL]

Looks like it's something about the currently used crazyflie_firmware. Have you encountered this? My current "solution" is just keep restarting the quad, click connect, and hoping the green LED shines after several times if I'm lucky.

To summarize:

1. Power loss seems inherent to CF, but we have workarounds (manual warmup)
2. Training works fine; deploy(build) needs native compile instructions (but Docker/native are similar)
3. Your cfclient is great and replaces trigger.py
4. Having new radio connection issues - any insights?

Really appreciate your amazing work—I’ve learned so much! Sorry again for the delayed response, and I’d love to hear your thoughts!