Skip to content

Moddable SDK 5.7.0

Choose a tag to compare

View all tags
@mkellner mkellner released this 13 May 04:50
· 256 commits to public since this release
5.7.0
2e7111b

Moddable SDK 5.7.0 contains improvements made between April 4, 2025 and May 12, 2025.

Realtime AI

Moddable continues to grow and enhance our AI support through our RealTime AI initiative. This month brings new integration of a new service, significant new features in our ConversationalAI application, and a simple new example to get you started adding AI agents to your embedded projects.

Eleven Labs integration

We updated our ChatAudioIO architecture with support for the latest ElevenLabs API, coincidentally named Conversational AI. The Moddable SDK has long supported streaming text-to-speech with ElevenLabs, and we find it has some of the most natural-sounding AI-generated speech out there. With our latest update, you can easily hear it for yourself and effortlessly compare it with the voice from other services. This support comes on the heels of our addition of HumeAI last month.

Choose AI service and voice in ConversationalAI application

Our amazing ConversationalAI application has been enhanced to allow the user to customize each AI agent by selecting an AI service to use (Hume, Google Gemini, OpenAI, or ElevenLabs) and the voice to use. This makes it incredibly easy to choose the service that works best for the agent and pair it with just the right voice. Your choices are saved in preferences, making them persistent across restarts.

To change the voice and service, tap the gear icon on any agent. The list of voices varies by service. For services that provide details about each voice, such as ElevenLabs, the UI shows them as tags, making it easy to find the kind of voice you are looking for.

User selecting AI service and voices in Conversational AI app

New example

Our conversationalAI example app wraps a beautiful, interactive user experience around our ChatAudioIO library. All that UI code makes it difficult to appreciate just how simple it is to add an interactive AI-powered voice chat to your project. To help with that, we have a new example that is about as small as possible. Here's the entire app – notice that most of it is event logging:

const chat = new ChatAudioIO({
	specifier: "humeAIEVI",
	voiceName: "Sunny",
	instructions: "You're a hostile fisherman with a salty sense of humor. You dislike people and care even less for fish.",
	onStateChanged(state) {
		trace(`State: ${ChatAudioIO.states[state]} ${this.error ?? ""}\n`);
	},
	onInputTranscript(text) {
		trace(`User: ${text}\n`);
	},
	onOutputTranscript(text) {
		trace(`Agent: ${text}\n`);
	},
});
chat.connect();

And it works with Hume, Google Gemini, OpenAI, and ElevenLabs.

ESP-IDF I²C upgrade

Espressif has been in the process of migrating to a new I²C API called i2c_master. The previous and new APIs cannot coexist in the same runtime, and the ESP Camera support uses the newer API. Consequently, we have migrated all I²C support to the new i2c_master API, including ECMA-419 I²C and the original pins I²C. The new API has some advantages, particularly when working with multiple devices simultaneously. However, it is not 100% functionally compatible. Of note, its support for stop bit handling and the read and write quick commands is limited. It appears this may be addressed in ESP-IDF v5.5.

We do not expect major issues in the transition; however some incompatibilities are possible given how widely used I²C is and how many variations there are. If you run into issues, please report them.

Node-RED MCU Edition

This release fixes a long-standing bug in the make system that prevented the Node-RED MCU Edition plugin from working with devices connected by native USB to macOS and Linux. This prevented many ESP32-S3 devices from working. This has been resolved.

In addition, fixes to the nRF52 build on Windows will soon allow nRF52-=powered devices to work with Node-RED MCU Edition on Windows as well!

TypeScript upgrade

We've updated our TypeScript support to default to ECMAScript 2024. That allows you to use more recent JavaScript language features, such as resizable ArrayBuffer, in your TypeScript projects. This should not have any compatibility issues, unless you had manually added type defintions for these functions to your project.

ESP-IDF v5.4

Moddable SDK 5.5 migrated ESP32 support to ESP-IDF v5.4 last month. If you develop for the ESP32 family, you'll need to update your ESP-IDF. Instructions are in last month's release notes.

Release Details

  • Contributed
    • conversationalAI
      • Select AI service in UI
      • Select agent voice in UI
  • Modules
    • ChatAudioIO
      • Add support for ElevenLabs service
      • Hume supports selecting voice by name
      • Rename humeAPIKey to humeAIKey for consistency #1481 (reported by @stc1988)
      • Fix sendText
      • sendFunctionResult now requires function name argument too
      • New CONNECTED state to know when service is fully ready to use
      • Gemini Live now supports live output transcription (but not yet input – waiting on Google)
      • Add array of state names at ChatAudioIO.states for debugging / logging
    • Timer
      • Remove obsolete modTimerGetID
      • Timer.clear() never throws, consistent with web platform's clearTimeout (useful for emulating web APIs)
    • Commodetto Outline
      • Handle stroke line weight as number or integer
      • Handle SVG z command
      • transform() method
    • Piu Outline – avoid unnecessary recalculation of outline bounds
    • Web Storage - new module provides WebStorage localStorage using ECMA-419 3rd Edition's embedded_storage/key-value
  • ECMA-419
    • UDP
      • Support multicast (experimental for 4th Edition) on macOS and lwip (ESP32)
      • Eliminates race condition that could stall incoming packets on lwip implementation
      • First argument to write is buffer, not remote address. (reported by @linfan68) #1487
    • HTTP Client now properly handles new request issued from onDone() callback
    • fetch() module supports PATCH method and works with EventSource
    • EventSource module supports body length of 0
    • HTTP Server now handles HTTP 101 response correctly by assuming respond body length of 0. This fixes WebSocket server support. (reported by @cmidgley) #1491
    • Audio In and Out allow repeated start() and stop()
  • Devices
    • ESP32
      • Add support for M5Atom Lite Echo Base, M5Atom S3 Echo Base (contributed by @kitazaki)
      • Add support for M5AtomS3R (contributed by @stc1988)
      • I²C now uses newer i2c_master_* APIs
      • Camera
        • Image controls framework (brightness, contrast, etc) for ESP32 and macOS (experimental for 4th Edition)
        • No longer deactivates I²C (not needed with i2c_master_* APIs)
      • ILI9341_P8 driver provides frameRate getter
      • More fixes for lack of GCC atomics on RISC-V targets
      • fxAbort logs error message on unhandled exception
    • nRF52
  • XS JavaScript Engine
    • New xsmcSetStringX() and xsSetStringX() to eliminate string copies
    • Fix RegExp with empty disfunction #1484 (reported by @gibson042)
    • Use C_* macros consistently
    • Include fx_iterator_from in snapshot table of native functions
    • Fix crash in mutabilites() with huge multibyte function names
    • JSX support now handles - in attribute name
  • TypeScript
    • Bumped to ES2024 (latest available) to get new features like resizable ArrayBuffer
    • Typings for ChatAudioIO
    • Rename embedded:audio typings to be consistent with standard
  • Examples
    • New simple ChatAudioIO app – great starting point for your projects
    • ECMA-419 httpserver example needs to use server.port for EventSource connection
  • Tools
    • cmake files for ESP32 build updated to quiet warnings about minimum version
    • More parallelism in ESP32 build on macOS & Linux
    • Set XSBUG_HOST and XSBUG_PORT on macOS & Linux when launching serial2xsbug on USB-native devices (e.g. ESP32-S3). Fixes Node-RED MCU Edition plugin integration for these devices. (reported by @mshioji)
    • Fix crash in clearTimer in xst (reported by @ChALkeR) #1490
    • Add TypeScript's dst directory to .gitignore list (contributed by @stc1988)
    • Workaround idf.py add_dependency now failing if dependency already exists
    • Add XSBUG_LOG_PORT to allow custom port on xsbug-log (contributed by @cmidgley) #1376

Contact Us

If you have questions or suggestions about anything here, please reach out:

Contributors

ChALkeR, gibson042, and 6 other contributors
Assets 20
Loading