Read from read loop remove generator logic

[brianignacio5]

Fix #224

Description

Using yield and async generator is not efficient logic for reading the serial port.

This PR introduces readloop as an always reading to a buffer and SLIP parsing from buffer. Seems to be more reliable.

This pull request refactors the serial communication logic to simplify data reading and improve reliability. The main changes focus on removing asynchronous generators in favor of simpler async functions, introducing a new sleep utility, and enhancing debugging and buffer management. The updates affect both the esploader.ts and webserial.ts files, resulting in more maintainable and understandable code.

Serial Communication Refactor:

• Replaced the asynchronous generator-based read logic in Transport with a simpler async function, removing newRead and generator-based readLoop. The new read method now waits for data with a timeout and processes available bytes in a straightforward loop. (src/webserial.ts) [1] [2]
• Updated all usages of transport.read().next() and transport.newRead() in ESPLoader to use the new transport.read() interface, simplifying packet reading and improving consistency. (src/esploader.ts) [1] [2] [3] [4]

Utility and Buffer Management:

• Added a new sleep utility function in util.ts and replaced all internal _sleep usages with this shared function. (src/util.ts, src/esploader.ts, src/webserial.ts) [1] [2] [3] [4]
• Added a peek() method to Transport for safely inspecting the buffer contents without consuming data, improving boot log detection logic. (src/webserial.ts, src/esploader.ts) [1] [2]

Debugging and Logging Enhancements:

• Added detailed debug logs for register reads and boot/download mode detection to aid troubleshooting and development. (src/esploader.ts) [1] [2]

Bug Fixes and Logic Improvements:

• Improved SLIP packet parsing: after a packet is found, any remaining bytes are now correctly preserved in the buffer for future reads. (src/webserial.ts) [1] [2] [3]
• Updated regular expression for boot log detection to more reliably match multiline logs. (src/esploader.ts)

These changes collectively make the serial communication code easier to understand, more robust, and better suited for debugging and extension.

Testing

Manual testing using examples/typescript commands.

---

Checklist

Before submitting a Pull Request, please ensure the following:

• 🚨 This PR does not introduce breaking changes.
• All CI checks (GH Actions) pass.
• Documentation is updated as needed.
• Tests are updated or added as necessary.
• Code is well-commented, especially in complex areas.
• Git history is clean — commits are squashed to the minimum necessary.

[github-actions]

Download the artifacts for this pull request:

• esptool-js-0.5.7.tgz.zip

[terrafirma2021]

can confirm fixed on Chrome Version 143.0.7499.110

[tyeth]

yeah seems fine on an s2 with same chrome version win11.
Haven't got another board with me to test, but will verify one of the older ones later (don't wait on me).
The connect button doesn't function after a failed attempt (initial connection from tinyusb msc+serial mode). However if you connect the console after the failure and then stop it, then connect program button functions again. The error situation needs cleanup on the transport/serial port.

[MitchBradley]

I tried this patch on my problem ESP32-S3. Unfortunately, it does not work. The failure is that the code fails to detect the S3s entry to Waiting for download state, even though the S3 really did enter that state and issued the appropriate message. Then the serial port gets a bunch of Buffer Overflows and never recovers. I think it is a race condition between the serial port setup and the timing of the messages.

This branch https://github.com/MitchBradley/esptool-js/tree/timedRead works with all of the ESP32s and ESP32-S3s that I have tested so far.

Every read goes through a low-level "timedRead()" function whose timeout ensures that the reader cannot stay locked forever. It retries all possible non-fatal errors that cause the loss of the reader, by getting a new reader per the example code in core WebSerial documentation. Fatal errors that cannot be recovered cause a throw.

The SLIP reader and a new "readLine(timeout)" function are built on top of a buffering layer that sits above timedRead(). readLine() is used for parsing/detecting the entry to download mode, thus avoiding a regex that spans multiple lines, and avoiding the possible race condition when those initial bytes were read all at once.

[brianignacio5]

Hi @MitchBradley Could you share which board are you using ? Using ESP32-S3-DevKitC-1 seems to work for me with this branch but maybe 3rd party boards would work differently. Also does it happen when using baud rate 115200 ?

Your approach is closer to the previous implementation without async generators which is cool but I have no way to test these changes unfortunately as I can't reproduce myself with Espressif boards.