feat(webcanvas): add method to obtain software renderer image data

[zubiden]

This PR makes the canvas selector optional for the software renderer. Instead, the rendering result can be obtained by calling the canvas.getFrameData() method. This allows the library to be run in a worker and draw onto an OffscreenCanvas.

To enhance type safety and enable potential future API expansions, this PR splits Canvas into three separate, engine-dependent subclasses. Currently, only SwCanvas has additional logic. The init function now infers the renderer type and provides the correct canvas subclass.

I was able to successfully pass canvas ownership to a worker, initialize the library there, and play the animation in my own tests.

animation.play((frame) => {
  canvas.update();
  canvas.render();

  const imageData = canvas.getFrameData();
  if (imageData) ctx.putImageData(imageData, 0, 0);
});

Let me know if API shape is good. I'll update the documentation and remove the part of my previous pr that fixed the Windows build (if it hadn't been merged by that time).

[zubiden]

Known issues:

• enableDevicePixelRatio must be disabled to work in worker. Slightly out of scope of this PR but needs to be addressed

[tinyjin]

@zubiden I haven't fully tested in WebWorker environment yet. But you can disable devicePixelRatio by setting enableDevicePixelRatio to false

enableDevicePixelRatio must be disabled to work in worker. Slightly out of scope of this PR but needs to be addressed

If you have reproducible issue with the workers, appreciate to have issue. (Please raise an issue with details)

[tinyjin]

@zubiden Just curious, is there critical issue from passing canvas selector to software engine?

[zubiden]

@zubiden Just curious, is there critical issue from passing canvas selector to software engine?

No, ability to do so is still there. This PR focuses on providing similar API to RLottie for a migration in my project.
Current implementation seems to be running synchronously on a main thread, which can block UI interactions.

[github-actions]

🚀 Playground preview deployment ready!

🎮 Playground: https://thorvg-playground-1ypmpnjyi-thorvg-web.vercel.app

[tinyjin]

@zubiden Just curious, is there critical issue from passing canvas selector to software engine?

No, ability to do so is still there. This PR focuses on providing similar API to RLottie for a migration in my project. Current implementation seems to be running synchronously on a main thread, which can block UI interactions.

@zubiden If so, I think you can just apply this patch to your project.
We're not aiming to provide similar interface with rLottie. Also the offcanvas rendering(e.g, to prevent UI blocking) as well as multi-threading isn't considered yet. Those topics should be discussed and reviewed more carefully later.

[hermet]

@zubiden Just curious, is there critical issue from passing canvas selector to software engine?

No, ability to do so is still there. This PR focuses on providing similar API to RLottie for a migration in my project. Current implementation seems to be running synchronously on a main thread, which can block UI interactions.

@zubiden If so, I think you can just apply this patch to your project. We're not aiming to provide similar interface with rLottie. Also the offcanvas rendering(e.g, to prevent UI blocking) as well as multi-threading isn't considered yet. Those topics should be discussed and reviewed more carefully later.

@tinyjin What is downside?

[zubiden]

In a nutshell, this PR adds the ability to skip the canvas and obtain the result of the software renderer directly as an ImageData object. This unlocks advanced usage patterns like Web Workers and offscreen canvases.

I've kept the current API the same. The new addition just allows you to skip the DOM selector for the SW renderer.

The Canvas subclasses were added to enhance the developer experience since the getFrameData method is unavailable for the WebGL and WebGPU implementations. It would be strange to offer this method on all engines, but it always returns undefined or throws an error on two of them.

[hermet]

@tinyjin async & offscreen rendering is useful for various scenarios. Please share your concern points if you have, Otherwise I think it's good to review properly. Thanks.

[tinyjin]

Just curious, is there critical issue from passing canvas selector to software engine?

No, ability to do so is still there

this PR adds the ability to skip the canvas and obtain the result of the software renderer directly as an ImageData object.

@zubiden Just to clarify:

I guess the current situation is that, in a Web Worker environment, accessing document would be blocked.
So assuming such blocking exists, this PR seems to focus on preventing errors caused by that restriction.

That said, I’m still wondering about the main motivation, since you mentioned earlier that there is no critical issue with passing a canvas selector.

From what I see, this PR mainly avoids touching the document API by allowing selector to be undefined, and to obtain the result of the software renderer directly as an ImageData object is what happens in application side?

[zubiden]

you mentioned earlier that there is no critical issue with passing a canvas selector

I thought you were asking if I had encountered a critical issue with canvas selector logic that required an immediate patching 😅

My motivation was to introduce a method of manually working with a rendered frame, independent of a DOM canvas. This library already works well in a web worker when using the SW engine, except for the previously mentioned issue with DPR.

As someone trying to migrate from RLottie, this is the last missing link to allow the same usage patterns.

[zubiden]

The ability to use any rendering engine inside the web worker is outside the scope of this PR. I opened an issue with some preliminary research, but Emscripten is not yet adapted for such usage.
Passing a selector while inside a worker will throw an error. I plan to document this if we proceed with this pull request.

[tinyjin]

Thanks, please check comments.

[github-actions]

🚀 Playground preview deployment ready!

🎮 Playground: https://thorvg-playground-h7ecek9hk-thorvg-web.vercel.app

[tinyjin]

Thanks. Minor nitpicks

[github-actions]

🚀 Playground preview deployment ready!

🎮 Playground: https://thorvg-playground-o5d6bii63-thorvg-web.vercel.app

[tinyjin]

Thank you for contribution :)

[tinyjin]

@zubiden Could you squash commits to leave necessary histories?
You may wanna add your name into CONTRIBUTORS (real name preferred )

[zubiden]

You may wanna add your name into CONTRIBUTORS (real name preferred )

Thanks, but I don't see the need. I'll create another pull request to add the DPR parameter and explore the potential memory leak when destroying the canvas (maybe I'm just doing it wrong 🤔). Unless I stumble upon another bug or missing features, that's probably all contributions from me.

I am quite happy with the current functionality. Great job!

[tinyjin]

@zubiden Thanks! Last. May I ask your correct email?

[github-actions]

🚀 Playground preview deployment ready!

🎮 Playground: https://thorvg-playground-kklaf33xl-thorvg-web.vercel.app

[hermet]

@zubiden @tinyjin Couldn't return the rendered image if the canvas already have?

[zubiden]

I didn't quite understand, but if you're asking about library drawing to a canvas element and then user calling getFrameData, that should work.

However, the underlying ArrayBuffer is shared. I can add cloning if needed.

[zubiden]

Sorry for the delay. Apparently, I added the comment as a PR review and then forgot to send it.

[zubiden]

I didn't quite understand, but if you're asking about library drawing to a canvas element and then user calling getFrameData, that should work.

However, the underlying ArrayBuffer is shared. I can add cloning if needed.

[zubiden]

@zubiden Thanks! Last. May I ask your correct email?

I prefer to use GitHub address to avoid spam from commit scanning bots. However, if you want to include somewhere for the future contact, it is zubiden.ua@gmail.com

[zubiden]

Just checking. Do I need to change anything to get this PR merged?

[hermet]

Just checking. Do I need to change anything to get this PR merged?

@zubiden We're reviewing the design part a bit more. Please wait for the submit. Thanks.