Using rp2040 as a keyboard AND a mouse through micropython

[dov]

I'm working on writing a micropython program for the the Pico W to make it into a websocket controlled USB-keyboard and mouse. I'm using the the micropython-lib packages usb-device-keyboard and usb-device-mouse. The keyboard part now seem to be working and I wanted to add mouse support as well. But when initializing the mouse, the keyboard part stops working.

Here is how to reproduce the problem.

1. Install the dependent packages:

mpremote connect /dev/ttyACM0 mip install usb-device-mouse
mpremote connect /dev/ttyACM0 mip install usb-device-keyboard

2. Add the following to main.py:

import usb.device
from usb.device.keyboard import KeyboardInterface, KeyCode, LEDCode
from usb.device.mouse import MouseInterface
import time

time.sleep_ms(5000) # Sleep so that we can interrupt at start

kb = KeyboardInterface()
mouse = MouseInterface()
usb.device.get().init(kb, builtin_driver=True)
# usb.device.get().init(mouse, builtin_driver=True)   # <--- If I do this init, I can no longer send keyboard events!

# Send the character 'a' twice a second
while True:
    kb.send_keys([KeyCode.A])  # <--- This is only received if I don't init the mouse
    kb.send_keys([])
    time.sleep_ms(500)

3. Restart the MCU and wait for the incoming 'a' characters
4. Change main.py and uncomment the mouse initialization
5. Redo and verify that there are no incoming 'a' characters.

So my question is, how can I initialize both the keyboard AND the mouse?

My version of micropython is:

MicroPython v1.25.0 on 2025-04-15; Raspberry Pi Pico W with RP2040

[dov]

See also:

micropython/micropython-lib#921

I also copied in this question into github copilot, and the response that I got was more or less that it is currently not supported in micropython, and it offered to help me write a combined keyboard and mouse module.

Here's github copilot's summary:

In summary:

You cannot implement a true composite Keyboard+Mouse HID device for USB on RP2040 MicroPython using only Python and the current micropython-lib. It requires C-level changes or a custom firmware build.

Would you like guidance on how to approach the C-level changes, or are you interested in workarounds or alternative approaches?

So it doesn't look like it is possible to do my project in micropython at this time. :-(

[Cerothen]

I was taking a look at this project and found the following solution to work for my testing purposes. The LED assignments are just so that I can visually see what the current HID device is.

Basically we keep the current state of the HID device, then swap to the other one if needed which seems to work well enough in this example.

# Reference Material
# https://github.com/orgs/micropython/discussions/17475
# https://github.com/micropython/micropython-lib/blob/fbf7e120c6830d8d04097309e715bcab63dcca67/micropython/usb/examples/device/keyboard_example.py
# https://github.com/micropython/micropython-lib/blob/fbf7e120c6830d8d04097309e715bcab63dcca67/micropython/usb/examples/device/mouse_example.py


import usb.device
from usb.device.keyboard import KeyboardInterface, KeyCode, LEDCode
from usb.device.mouse import MouseInterface
import time
from machine import Pin

# Debugging Status Indication
mouseLED = Pin(15, Pin.OUT) # BLUE
keyboardLED = Pin(8, Pin.OUT) # GREEN

# Default LED States
mouseLED.on()
keyboardLED.off()

lastDevice = 0 # 0-NONE, 1-KEYBOARD, 2-MOUSE
defaultDelay = 150

print("Initializing")
for i in range(10):
    print(i)
    mouseLED.toggle()
    keyboardLED.toggle()
    time.sleep_ms(500)

# Default LED States
mouseLED.off()
keyboardLED.off()

print("Device Setup")
kb = KeyboardInterface()
mouse = MouseInterface()

def mouse_init():
    # Initialize Mouse If Needed
    global lastDevice
    if lastDevice != 2:
        usb.device.get().init(mouse, builtin_driver=True)
        while not mouse.is_open():
            time.sleep_ms(100)
        time.sleep_ms(250)
        lastDevice = 2
    keyboardLED.off()
    mouseLED.on()
        
def kb_init():
    # Initialize Mouse If Needed
    global lastDevice
    if lastDevice != 1:
        usb.device.get().init(kb, builtin_driver=True)
        while not kb.is_open():
            time.sleep_ms(100)
        time.sleep_ms(250)
        lastDevice = 1
    mouseLED.off()
    keyboardLED.on()

def mouse_left_click(leftRight=1, hold=0):
    mouse_init()
    if leftRight == 0:
        mouse.click_left(False)
        mouse.click_right(False)
    elif leftRight == 1:
        mouse.click_left(True)
        time.sleep_ms(100)
        if hold == 0:
            mouse.click_left(False)
    elif leftRight == 2:
        mouse.click_right(True)
        time.sleep_ms(100)
        if hold == 0:
            mouse.click_right(False)
    time.sleep_ms(defaultDelay)

def mouse_move(posX, posY):
    mouse_init()
    mouse.move_by(posX, posY)
    time.sleep_ms(defaultDelay)

def send_key(keyVal):
    kb_init()
    kb.send_keys([keyVal])
    kb.send_keys([])
    time.sleep_ms(defaultDelay)


mouse_move(-100,0)
mouse_move(0,100)
mouse_move(100,0)
mouse_move(0,-100)

send_key(KeyCode.A)
send_key(KeyCode.B)
send_key(KeyCode.C)
send_key(KeyCode.D)

mouse_move(-100,0)
mouse_move(0,100)
mouse_move(100,0)
mouse_move(0,-100)

send_key(KeyCode.A)
send_key(KeyCode.B)
send_key(KeyCode.C)
send_key(KeyCode.D)

[Cerothen]

After considerable digging and hating myself for contemplating this project I would like to provide anyone that is interested in where I landed on this to help others out.

rp2040_hid.py (PLACE IN LIB)

# rp2040_hid.py
#
# MicroPython composite USB HID device for RP2040
#
# Features:
#   - Absolute mouse
#   - Mouse wheel
#   - Left/right/middle click
#   - Click hold/release
#   - Keyboard
#   - Media keys
#   - Caps/Num/Scroll lock state
#   - Special key name resolution
#   - String typing with {SPECIAL_KEYS}
#
# Requires:
#   MicroPython with usb.device enabled
#
# ============================================================

import struct
import machine
import usb.device
from usb.device.hid import HIDInterface


# ============================================================
# HID REPORT DESCRIPTOR
# ============================================================

HID_REPORT_DESCRIPTOR = bytes([

    # ========================================================
    # MOUSE
    # Report ID 1
    # ========================================================

    0x05, 0x01,              # Usage Page (Generic Desktop)
    0x09, 0x02,              # Usage (Mouse)
    0xA1, 0x01,              # Collection (Application)

    0x85, 0x01,              # Report ID (1)

    0x09, 0x01,              # Usage (Pointer)
    0xA1, 0x00,              # Collection (Physical)

    # Buttons
    0x05, 0x09,
    0x19, 0x01,
    0x29, 0x03,              # 3 buttons
    0x15, 0x00,
    0x25, 0x01,
    0x95, 0x03,
    0x75, 0x01,
    0x81, 0x02,

    # Padding
    0x95, 0x05,
    0x75, 0x01,
    0x81, 0x03,

    # Absolute X/Y
    0x05, 0x01,
    0x09, 0x30,
    0x09, 0x31,

    0x16, 0x00, 0x00,
    0x26, 0xFF, 0x7F,

    0x75, 0x10,
    0x95, 0x02,
    0x81, 0x02,

    # Wheel
    0x09, 0x38,
    0x15, 0x81,
    0x25, 0x7F,
    0x75, 0x08,
    0x95, 0x01,
    0x81, 0x06,

    0xC0,
    0xC0,

    # ========================================================
    # KEYBOARD
    # Report ID 2
    # ========================================================

    0x05, 0x01,
    0x09, 0x06,
    0xA1, 0x01,

    0x85, 0x02,

    # Modifiers
    0x05, 0x07,
    0x19, 0xE0,
    0x29, 0xE7,
    0x15, 0x00,
    0x25, 0x01,
    0x75, 0x01,
    0x95, 0x08,
    0x81, 0x02,

    # Reserved
    0x75, 0x08,
    0x95, 0x01,
    0x81, 0x03,

    # Keys
    0x15, 0x00,
    0x25, 0xDD,
    0x05, 0x07,
    0x19, 0x00,
    0x29, 0xDD,
    0x75, 0x08,
    0x95, 0x06,
    0x81, 0x00,

    # LED output
    0x05, 0x08,
    0x19, 0x01,
    0x29, 0x05,
    0x75, 0x01,
    0x95, 0x05,
    0x91, 0x02,

    # LED padding
    0x75, 0x03,
    0x95, 0x01,
    0x91, 0x03,

    0xC0,

    # ========================================================
    # MEDIA KEYS
    # Report ID 3
    # ========================================================

    0x05, 0x0C,              # Consumer Page
    0x09, 0x01,              # Consumer Control
    0xA1, 0x01,

    0x85, 0x03,              # Report ID 3

    0x15, 0x00,
    0x26, 0xFF, 0x03,

    0x19, 0x00,
    0x2A, 0xFF, 0x03,

    0x75, 0x10,
    0x95, 0x01,

    0x81, 0x00,

    0xC0
])


# ============================================================
# HID CLASS
# ============================================================

class RP2040HID(HIDInterface):

    # ========================================================
    # REPORT IDS
    # ========================================================

    REPORT_ID_MOUSE = 1
    REPORT_ID_KEYBOARD = 2
    REPORT_ID_MEDIA = 3

    # ========================================================
    # MOUSE BUTTONS
    # ========================================================

    BUTTON_LEFT = 0x01
    BUTTON_RIGHT = 0x02
    BUTTON_MIDDLE = 0x04

    # ========================================================
    # MODIFIERS
    # ========================================================

    MOD_LCTRL  = 0x01
    MOD_LSHIFT = 0x02
    MOD_LALT   = 0x04
    MOD_LGUI   = 0x08

    MOD_RCTRL  = 0x10
    MOD_RSHIFT = 0x20
    MOD_RALT   = 0x40
    MOD_RGUI   = 0x80

    # ========================================================
    # LEDS
    # ========================================================

    LED_NUM_LOCK = 0x01
    LED_CAPS_LOCK = 0x02
    LED_SCROLL_LOCK = 0x04

    # ========================================================
    # MEDIA KEYS
    # ========================================================

    MEDIA_PLAY_PAUSE = 0x00CD
    MEDIA_STOP = 0x00B7
    MEDIA_NEXT = 0x00B5
    MEDIA_PREVIOUS = 0x00B6
    MEDIA_VOLUME_UP = 0x00E9
    MEDIA_VOLUME_DOWN = 0x00EA
    MEDIA_MUTE = 0x00E2

    # ========================================================
    # SPECIAL KEY MAP
    # ========================================================

    KEYMAP = {

        # Control
        "ENTER": 0x28,
        "ESCAPE": 0x29,
        "BACKSPACE": 0x2A,
        "TAB": 0x2B,
        "SPACE": 0x2C,

        # Navigation
        "INSERT": 0x49,
        "HOME": 0x4A,
        "PAGEUP": 0x4B,
        "DELETE": 0x4C,
        "END": 0x4D,
        "PAGEDOWN": 0x4E,

        # Arrows
        "RIGHT": 0x4F,
        "LEFT": 0x50,
        "DOWN": 0x51,
        "UP": 0x52,

        # Locks
        "CAPSLOCK": 0x39,
        "NUMLOCK": 0x53,
        "SCROLLLOCK": 0x47,

        # Print screen
        "PRINTSCREEN": 0x46,

        # Function keys
        "F1": 0x3A,
        "F2": 0x3B,
        "F3": 0x3C,
        "F4": 0x3D,
        "F5": 0x3E,
        "F6": 0x3F,
        "F7": 0x40,
        "F8": 0x41,
        "F9": 0x42,
        "F10": 0x43,
        "F11": 0x44,
        "F12": 0x45,

        # Left modifiers
        "LCTRL": 0xE0,
        "LSHIFT": 0xE1,
        "LALT": 0xE2,
        "LGUI": 0xE3,

        # Right modifiers
        "RCTRL": 0xE4,
        "RSHIFT": 0xE5,
        "RALT": 0xE6,
        "RGUI": 0xE7,
    }

    # ========================================================
    # INIT
    # ========================================================

    def __init__(self):

        super().__init__(
            report_descriptor=HID_REPORT_DESCRIPTOR,
            extra_descriptors=b"",
            set_report_buf=bytearray(2),
        )

        self.led_state = 0

        self.mouse_buttons = 0
        self.mouse_x = 0
        self.mouse_y = 0

    # ========================================================
    # HOST OUTPUT REPORT
    # ========================================================

    def on_set_report(self, report_data, report_id, report_type):
        if report_id == self.REPORT_ID_KEYBOARD:
            if len(report_data) >= 1:
                self.led_state = report_data[0]

    # ========================================================
    # MOUSE
    # ========================================================

    def _send_mouse_report(self, wheel=0):
        report = struct.pack(
            "<BBHHb",
            self.REPORT_ID_MOUSE,
            self.mouse_buttons,
            self.mouse_x,
            self.mouse_y,
            wheel
        )
        self.send_report(report)

    def move_mouse(self, x, y):
        x = max(0, min(32767, x))
        y = max(0, min(32767, y))
        self.mouse_x = x
        self.mouse_y = y
        self._send_mouse_report()

    def scroll_mouse(self, amount):
        if amount < -127:
            amount = -127
        if amount > 127:
            amount = 127

        self._send_mouse_report(amount)

    # ========================================================
    # BUTTON HOLD/RELEASE
    # ========================================================

    def hold_left_button(self):
        self.mouse_buttons |= self.BUTTON_LEFT
        self._send_mouse_report()

    def release_left_button(self):
        self.mouse_buttons &= ~self.BUTTON_LEFT
        self._send_mouse_report()

    def hold_right_button(self):
        self.mouse_buttons |= self.BUTTON_RIGHT
        self._send_mouse_report()

    def release_right_button(self):
        self.mouse_buttons &= ~self.BUTTON_RIGHT
        self._send_mouse_report()

    def hold_middle_button(self):
        self.mouse_buttons |= self.BUTTON_MIDDLE
        self._send_mouse_report()

    def release_middle_button(self):
        self.mouse_buttons &= ~self.BUTTON_MIDDLE
        self._send_mouse_report()

    # ========================================================
    # CLICKS
    # ========================================================

    def left_click(self):
        self.hold_left_button()
        machine.idle()
        self.release_left_button()

    def right_click(self):
        self.hold_right_button()
        machine.idle()
        self.release_right_button()

    def middle_click(self):
        self.hold_middle_button()
        machine.idle()
        self.release_middle_button()

    # ========================================================
    # KEYBOARD
    # ========================================================

    def send_keyboard_report(self, modifiers=0, keys=None):
        if keys is None:
            keys = []

        keys = keys[:6]

        while len(keys) < 6:
            keys.append(0)

        report = struct.pack(
            "<BBB6B",
            self.REPORT_ID_KEYBOARD,
            modifiers,
            0,
            *keys
        )

        self.send_report(report)

    def key_press(self, keycode, modifiers=0):
        self.send_keyboard_report(modifiers, [keycode])

    def key_release(self):
        self.send_keyboard_report()

    def key_tap(self, keycode,  modifiers=0):
        self.key_press(keycode, modifiers)
        machine.idle()
        self.key_release()

    # ========================================================
    # SPECIAL KEYS
    # ========================================================

    def resolve_special_key(self, name):
        return self.KEYMAP.get(name.upper(), None)

    def special_key_press(self, name, modifiers=0):
        kc = self.resolve_special_key(name)
        if kc is not None:
            self.key_press(kc, modifiers)

    def special_key_release(self):
        self.key_release()

    def special_key_tap(self, name, modifiers=0):
        kc = self.resolve_special_key(name)
        if kc is not None:
            self.key_tap(kc, modifiers)

    # ========================================================
    # MEDIA KEYS
    # ========================================================

    def send_media_key(self, usage):
        report = struct.pack(
            "<BH",
            self.REPORT_ID_MEDIA,
            usage
        )
        self.send_report(report)
        machine.idle()
        report = struct.pack(
            "<BH",
            self.REPORT_ID_MEDIA,
            0
        )
        self.send_report(report)

    def media_play_pause(self):
        self.send_media_key(self.MEDIA_PLAY_PAUSE)

    def media_stop(self):
        self.send_media_key(self.MEDIA_STOP)

    def media_next(self):
        self.send_media_key(self.MEDIA_NEXT)

    def media_previous(self):
        self.send_media_key(self.MEDIA_PREVIOUS)

    def media_volume_up(self):
        self.send_media_key(self.MEDIA_VOLUME_UP)

    def media_volume_down(self):
        self.send_media_key(self.MEDIA_VOLUME_DOWN)

    def media_mute(self):
        self.send_media_key(self.MEDIA_MUTE)

    # ========================================================
    # LED STATE
    # ========================================================

    def caps_lock(self):
        return bool(
            self.led_state &
            self.LED_CAPS_LOCK
        )

    def num_lock(self):
        return bool(
            self.led_state &
            self.LED_NUM_LOCK
        )

    def scroll_lock(self):
        return bool(
            self.led_state &
            self.LED_SCROLL_LOCK
        )

    # ========================================================
    # ASCII -> HID
    # ========================================================

    def ascii_to_hid(self, c):
        if 'a' <= c <= 'z':
            return (
                ord(c) - ord('a') + 0x04,
                0
            )

        if 'A' <= c <= 'Z':
            return (
                ord(c) - ord('A') + 0x04,
                self.MOD_LSHIFT
            )

        if '1' <= c <= '9':
            return (
                ord(c) - ord('1') + 0x1E,
                0
            )

        if c == '0':
            return (0x27, 0)

        if c == ' ':
            return (0x2C, 0)

        if c == '\n':
            return (0x28, 0)

        symbols = {

            '-': (0x2D, 0),
            '=': (0x2E, 0),
            '[': (0x2F, 0),
            ']': (0x30, 0),
            '\\': (0x31, 0),
            ';': (0x33, 0),
            '\'': (0x34, 0),
            '`': (0x35, 0),
            ',': (0x36, 0),
            '.': (0x37, 0),
            '/': (0x38, 0),

            '!': (0x1E, self.MOD_LSHIFT),
            '@': (0x1F, self.MOD_LSHIFT),
            '#': (0x20, self.MOD_LSHIFT),
            '$': (0x21, self.MOD_LSHIFT),
            '%': (0x22, self.MOD_LSHIFT),
            '^': (0x23, self.MOD_LSHIFT),
            '&': (0x24, self.MOD_LSHIFT),
            '*': (0x25, self.MOD_LSHIFT),
            '(': (0x26, self.MOD_LSHIFT),
            ')': (0x27, self.MOD_LSHIFT),
        }

        return symbols.get(c, (0, 0))

    # ========================================================
    # TYPE STRING
    # ========================================================

    def type_string(self, text):

        i = 0
        while i < len(text):
            c = text[i]
            
            # Escaped {
            if c == '\\':
                if i + 1 < len(text):
                    next_c = text[i + 1]
                    
                    if next_c in ['{', '}', '\\']:
                        kc, mod = self.ascii_to_hid(next_c)
                        self.key_tap(kc, mod)
                        i += 2
                        continue

                kc, mod = self.ascii_to_hid(c)
                self.key_tap(kc, mod)

                i += 1
                continue

            # Special key
            if c == '{':
                end = text.find('}', i)
                if end != -1:
                    key_name = text[i + 1:end]
                    kc = self.resolve_special_key(key_name)

                    if kc is not None:
                        self.key_tap(kc)
                        i = end + 1
                        continue

            # Normal char
            kc, mod = self.ascii_to_hid(c)
            self.key_tap(kc, mod)
            i += 1

For testing I also have a serial library that I use on pins 0 and 1 with a CP2102 UART (once the HID initializes then I no longer could interact with the board so I wanted a way that I could send random commands to it.

rp2040_serial.py (PLACE IN LIB)

import time
import gc
from machine import Pin, UART


class SerialInterface:
    def __init__(
        self,
        uart_id=0,
        baudrate=115200,
        tx_pin=0,
        rx_pin=1,
        line_ending=b"\x0d",
        echo=True
    ):
        self.uart = UART(
            uart_id,
            baudrate=baudrate,
            tx=Pin(tx_pin),
            rx=Pin(rx_pin)
        )

        self.line_ending = line_ending
        self.echo = echo

        self.rx_buf = b""

        # Command dictionary:
        # "command": {
        #     "params": int,
        #     "func": callable
        # }
        self.command_dictionary = {}

        # Register built-in commands
        self.register_command("wt", 1, self.command_wait)
        self.register_command("info", 0, self.command_info)
        self.register_command("clear", 0, self.clear_terminal)

        self.clear_terminal()

    # ------------------------------------------------------------------
    # UART / Terminal Utilities
    # ------------------------------------------------------------------

    def clear_terminal(self):
        self.uart.write(b"\x1b[2J\x1b[H")

    def write(self, msg):
        try:
            self.uart.write(str(msg) + "\r\n")
        except Exception:
            return False
        return True

    # ------------------------------------------------------------------
    # Built-In Commands
    # ------------------------------------------------------------------

    def command_wait(self, length_ms):
        self.write("[serialCommandWait] Sleep MS: {}".format(length_ms))
        time.sleep_ms(int(length_ms))
        self.write("[serialCommandWait] Sleep MS: Complete")

    def command_info(self):
        self.write("[serialCommandInfo] Free: {}\tUsed: {}".format(gc.mem_free(), gc.mem_alloc()))

    # ------------------------------------------------------------------
    # Command Registration
    # ------------------------------------------------------------------

    def register_command(self, name, param_count, func):
        self.command_dictionary[name] = {
            "params": param_count,
            "func": func
        }

    def unregister_command(self, name):
        if name in self.command_dictionary:
            del self.command_dictionary[name]

    # ------------------------------------------------------------------
    # Command Execution
    # ------------------------------------------------------------------

    def execute_command(self, command_string):

        if not command_string:
            return False

        # --------------------------------------------------------------
        # Tokenizer
        # Supports:
        #   normal words
        #   "quoted strings"
        #   escaped quotes: \"
        #   escaped backslashes: \\
        # --------------------------------------------------------------

        parts = []

        current = ""
        in_quotes = False
        escape = False

        for ch in command_string:

            # Handle escaped characters
            if escape:

                if ch == '"' or ch == '\\':
                    current += ch
                else:
                    # Preserve unknown escapes literally
                    current += '\\' + ch

                escape = False
                continue

            # Escape start
            if ch == '\\':
                escape = True
                continue

            # Quote handling
            if ch == '"':

                in_quotes = not in_quotes

                # End quote -> push token
                if not in_quotes:
                    parts.append(current)
                    current = ""

                continue

            # Space delimiter (outside quotes)
            if ch.isspace() and not in_quotes:

                if current:
                    parts.append(current)
                    current = ""

                continue

            current += ch

        # Trailing escape
        if escape:
            current += '\\'

        # Unterminated quote
        if in_quotes:
            self.write("[serialCommand] Unterminated quoted string")
            return False

        # Push final token
        if current:
            parts.append(current)

        # --------------------------------------------------------------
        # Command Execution
        # --------------------------------------------------------------

        i = 0
        while i < len(parts):
            command_name = parts[i]

            if command_name not in self.command_dictionary:
                self.write("[serialCommand] Unknown Command:\n{}".format(" ".join(parts[i:])))
                return False

            # Setup Command Variables
            command_data = self.command_dictionary[command_name]
            param_count = command_data["params"]
            func = command_data["func"]
            params = parts[i + 1:i + 1 + param_count]

            # Validate parameter count
            if len(params) != param_count:
                self.write("[serialCommand] Invalid parameter count for '{}'".format(command_name))
                return False
            
            # Attempt to Execute
            try:
                func(*params)
            except Exception as e:
                self.write("[serialCommand] Error running '{}': {}".format(command_name, e ))

            i += 1 + param_count

        return True

    # ------------------------------------------------------------------
    # RX Processing
    # ------------------------------------------------------------------

    def process_rx(self):
        while self.uart.any():

            byte_data = self.uart.read(1)

            if not byte_data:
                continue

            # Small delay for UART stability
            time.sleep_ms(2)

            # ENTER / CARRIAGE RETURN
            if byte_data == self.line_ending:

                try:
                    command = self.rx_buf.decode("utf-8").strip()
                except Exception:
                    command = ""

                self.rx_buf = b""

                self.uart.write("\r\n")

                if command:
                    self.write("[RX] {}".format(command))
                    self.execute_command(command)

            # BACKSPACE or DELETE
            elif byte_data in (b'\x08', b'\x7f'):

                # Only backspace if buffer contains characters
                if len(self.rx_buf) > 0:

                    # Remove last character
                    self.rx_buf = self.rx_buf[:-1]

                    # Erase character on terminal:
                    # move back, print space, move back again
                    if self.echo:
                        self.uart.write(b'\x08 \x08')

            # NORMAL CHARACTER
            else:

                self.rx_buf += byte_data

                # Echo typed characters
                if self.echo:
                    self.uart.write(byte_data)

        time.sleep_ms(10)

Lastly my main.py which mostly just initializes some helper functions and registers them in the serial terminal. Keeping with tradition my dev board still has my blue and green LEDs...

# Standard Imports
import time
import math
from machine import Pin

# Initialize Serial Interface
from rp2040_serial import SerialInterface
serial = SerialInterface()

# Initialize Delay (to allow interrupt)
serial.write("[MAIN] Initialize Delay (6 Seconds)")
print("[MAIN] Initialize Delay (6 Seconds)")
time.sleep_ms(6000)

# Initialize Keyboard and Mouse HID
import usb.device
from rp2040_hid import RP2040HID
hid = RP2040HID()
usb.device.get().init(hid)

# Initialize LED Pins
blueLED = Pin(15, Pin.OUT) # BLUE
greenLED = Pin(8, Pin.OUT) # GREEN

# General Global Variables
screenWidth = 1920
screenHeight = 1080

# Define HID Serial Functions
def serialMouseMove(x,y):
    serial.write("[serialMouseMove] X: {} Y: {}".format(x,y))
    hid.move_mouse(int(x), int(y))
    
def serialSetScreenSize(width, height):
    serial.write("[serialSetScreenSize] Width: {} Height: {}".format(width,height))
    global screenWidth
    global screenHeight
    screenWidth = int(width)
    screenHeight = int(height)
    
def serialCoordMove(x,y):
    mappedX = math.floor((int(x) / screenWidth) * 32766) + 1
    mappedY = math.floor((int(y) / screenHeight) * 32766) + 1
    serial.write("[serialCoordMove] X: {} ({}) Y: {} ({})".format(x, mappedX, y, mappedY))
    hid.move_mouse(int(mappedX), int(mappedY))

def serialMouseLeftClick():
    serial.write("[serialMouseLeftClick] Left Mouse Click")
    hid.left_click()
    
def serialMouseLeftHold():
    serial.write("[serialMouseLeftHold] Left Mouse Hold")
    hid.hold_left_button()
    
def serialMouseLeftRelease():
    serial.write("[serialMouseLeftRelease] Left Mouse Release")
    hid.release_left_button()
    
def serialMouseRightClick():
    serial.write("[serialMouseRightClick] Right Mouse Click")
    hid.right_click()
    
def serialMouseRightHold():
    serial.write("[serialMouseRightHold] Right Mouse Hold")
    hid.hold_right_button()
    
def serialMouseRightRelease():
    serial.write("[serialMouseRightRelease] Right Mouse Release")
    hid.release_right_button()
    
def serialStringToKeyboard(text):
    serial.write("[serialStringToKeyboard] Send: {}".format(text))
    hid.type_string(text)
    
def serialBlueLED():
    serial.write("[serialBlueLED] Toggle")
    blueLED.toggle()

def serialGreenLED():
    serial.write("[serialGreenLED] Toggle")
    greenLED.toggle()

# Register Serial Commands
serial.register_command("mm", 2, serialMouseMove)
serial.register_command("ss", 2, serialSetScreenSize)
serial.register_command("cm", 2, serialCoordMove)
serial.register_command("lmc", 0, serialMouseLeftClick)
serial.register_command("lmh", 0, serialMouseLeftHold)
serial.register_command("lmr", 0, serialMouseLeftRelease)
serial.register_command("rmc", 0, serialMouseRightClick)
serial.register_command("rmh", 0, serialMouseRightHold)
serial.register_command("rmr", 0, serialMouseRightRelease)
serial.register_command("sk", 1, serialStringToKeyboard)
serial.register_command("blue", 0, serialBlueLED)
serial.register_command("green", 0, serialGreenLED)

# Main Loop
serial.write("[MAIN] Initialized, starting serial loop")
while True:
    serial.process_rx()

I like this for testing since it lets me send a command like wt 2000 sk "This is a test{TAB}string" cm 500 500 lmc to wait 2000 ms, then send a string with a special character, move the mouse to 500 500 on a screen that is 1920x1080 and send a left mouse click

Hopefully all these things help others.