AutoPilot

This is a little experiment to get a Lego RC car to drive itself. PyBricks gives us a new firmware so we can operate the machine.

The idea is to stick a phone on top of the car (build yourself a Lego holder, you can do it) and use its camera to drive around.

The repo includes two strategies under drive:

• keyboard: drive the car manually, maybe while using the camera in a first-person kind of way
• seek: trained to follow a specific target (during the initial experiments, this was a small stuffed monkey)

The long term idea for this was to add some mapping of my house so the car could roam around freely. Could still happen.

Running The Thing

To connect to the hub and drive around, run main.py from the console. The script will connect to the Technic Hub, provided it's already been equipped with the firmware.

So, first of all, install firmware/hub.py on the hub from PyBricks. This should be self-explanatory, but basically boils down to copy-pasting the script into PyBricks' interface and doing what it says. Once that's done:

1. Turn on the hub
2. Run python3 main.py and wait for the START message in the console
3. Tap the button in the hub again to start its program. It starts by moving the wheels left and right to calibrate, which also makes it easy to see if it's running.
4. Profit!

To swap

Training

The training/datasets/monkey folder contains the data used for training. The raw images are generated by extracting frames from a recording through VLC. To convert these raw frames into usable data, run:

cd training
python3 make_labels.py

Once the data is ready, we can train the model and save some weights:

cd training
python3 train.py

The best weights generated in during the run should be copied to the root folder as monkey_weights. They will be used by the rest of the program.