CanvasRenderingContext2D.prototype.drawImage is now asynchronous

[NAllred91]

YOU MUST read first!

Please use Community Forum for general technical discussions and questions.

• I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).
• I have provided versions of third party library name, ios, Xcode and plugin version and adapter.js version if used.

Note: If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue.

Versions affected

• Cordova version (e.g 7.1.0): 10.0.0
• Cordova iOS version (e.g 5.1.0): 6.2.0
• Plugin version (e.g 6.0.12): 6.0.20
• iOS version (e.g 10.2): 13
• Xcode version (e.g 11.1 - 11A1027): 12.4

Description

The following patch of CanvasRenderingContext2D.prototype.drawImage has made drawImage asynchronous. This presents a problem when you want to call getImageData immediately after drawImage. Using a setTimeout to call getImageData will kind of work, but it might be better if drawImage returned a promise after it has been patched?

Referenced comment

6.0.15 support CanvasRenderingContext2D.drawImage on VideoElement with iosrtc MediaStream

03e4a0d

// Apply CanvasRenderingContext2D.drawImage monkey patch
var drawImage = CanvasRenderingContext2D.prototype.drawImage;
CanvasRenderingContext2D.prototype.drawImage = function (arg) {
	var args = Array.prototype.slice.call(arguments);
	var context = this;
	if (arg instanceof HTMLVideoElement && arg.render) {
		arg.render.save(function (data) {
		    var img = new window.Image();
		    img.addEventListener("load", function () {
		    	args.splice(0, 1, img);
		        drawImage.apply(context, args);
		    });
		    img.setAttribute("src", "data:image/jpg;base64," + data);
	  	});
	} else {
		return drawImage.apply(context, args);
	}

Originally posted by @hthetiot in #116 (comment)

[hthetiot]

render.save is async, i dont see how i can make it sync unfortunately.

[NAllred91]

Sorry, I’m not asking for it to be made sync. It would just be nice if it returned a promise that resolved when it was done. That way I would be able to do something when the operation is complete, instead of relying on an arbitrary timeout.

[NAllred91]

The following is a little example of how you could return a promise

 if (arg instanceof HTMLVideoElement && arg.render) {
            return new Promise<void>((resolve, reject) => {
               ...[Removing code for clarity]...

                // Reject the promise if it doesn't complete in some amount of time.
                setTimeout(() => {
                    reject(new Error('Draw image timed out!'))
                }, 500)

                arg.render.save(function (data) {
                   
                    var onLoad = function () {
                        img.removeEventListener('load', onLoad);
                        drawImage.apply(context, args);
                        URL.revokeObjectURL(temp);

                        // Resolve the promise once the operation is complete.
                        resolve();
                    };
                    img.addEventListener('load', onLoad);
                    img.setAttribute('src', temp);
                });
            });
        } else {

Then, it makes something like this possible

await ctx.drawImage(this._video, 0, 0, this._video.videoWidth, this._video.videoHeight);
const data = ctx.getImageData(0, 0, 3, 3).data;
console.log('This is data that was just drawn to the canvas!', data)

Otherwise, you have to use an arbitrary setTimeout to call getImageData, and then you're just hoping that the draw has already happened.

[hthetiot]

No problem with promise, will do and keep you posted.

[hthetiot]

PR would be welcome @NAllred91