Implement an importer to support Lottie animation

[beicause]

Related proposals:

• Add support for Lottie animation using ThorVG godot-proposals#9104

• Bugsquad edit, closes: Add support for Lottie animation using ThorVG godot-proposals#9104

Edit:
This PR addes a Lottie resource importer using ThorVG to import Lottie as CompressedTexture of pre-rendered sprite sheet. It's a editor only feature and doesn't implement real time rendering.

[fire]

I'll put this on my list to review for the pipeline import.

[fire]

1. Run the godot engine binary with --doctool to update the documentation
2. I'll try to give you guidance for the "json" property being set as the default value

Here are the changes: beicause/godot@lottie...V-Sekai:godot:vsk-lottie-4.3

3. Thoughts on extending Ref<Image> directly? You could also extend Ref<Texture2D>.
4. There's a system to convert ".json" to "LottieSheet" that is missing that goes through the ResourceImporter.

[beicause]

Thanks for your guide.
I have changed it to inherit Texture2D. Now in editor it display Lottie normally, but when running the game it's empty. I don't find the reason.

[fire]

I was stuck, I suspect we need to use the resource importer system that takes json and gives a LottieTexture2D

[beicause]

I was stuck, I suspect we need to use the resource importer system that takes json and gives a LottieTexture2D

Don't open json directly. Create a LottieTexture2D and add the json as subresource.

[beicause]

I'm not very familiar with custom resource. I still can't understan why using code to set this texture works:

texture=LottieTexture2D.create_from_string(FileAccess.get_file_as_string("res://lottie.json"),1.0)

But setting the texture in editor doesn't work in games. ( Update: create_from_json doesn't work too. Fixed by disabling JSON::stringify sort_keys )

For resource importer, since lottie extension name is also .json and will be recognized as JSON, we have to customize one, for example .lottie. Or don't implement resource importer for Lottie, just create it manually and use JSON as property.

[fire]

So there's a way to create a resource importer plugin that scans for "*.json" as lottie.

I'll try to give you instructions as soon as I can or you can try it.

[Calinou]

This implementation is simple and just works for me, by generating animation frame on-demand.

While this is a simpler approach, note that it'll result in less smooth animations that no longer look crisp when zoomed in (e.g. due to Camera2D zoom or canvas_items stretch mode).

I think Lottie's biggest use case is that it allows for vector-based 2D animation with an arbitrary amount of frames (in other words, things can be freely interpolated). While you could choose to import at a high framerate (which doesn't seem to be adjustable in this importer right now), targeting 60 FPS will require a lot of memory.

This might be a dealbreaker for some use cases, so it's worth considering whether a true vector-based approach is more suitable first.

Performance is also an important caveat, similar to AnimatedTexture. Updating texture data often is slow, and it can also be inflexible (e.g. it might not obey pausing as expected).

On the bright side, this texture-based approach has some upsides, such as being easier to use in 3D (e.g. with Sprite3D). A vector-based solution wouldn't be usable in 3D unless rendered to a texture via ViewportTexture, in which case it would also lose its vector-based nature.

[beicause]

The common use case of Lottie is UI or other short duration animation, where AnimatedTexture is convenient. We can add a resource loader to import Lottie as AnimatedTexture
( I just find AnimatedTexture is deprecated, so it's not a good solution. )

[hermet]

Hello, Texture (or AnimatedTexture) methods have several advantages.

Primarily, GPU with direct drawing is not always good & faster since the following factors:

• When using Lottie as a static resource like SVG, or when animations are paused, the result from one rendering can be quickly reused in subsequent frames. This is particularly evident when using Lottie with UI assets that rarely change resolution.
• Lottie requires more composition steps than expected to complete a scene. This process involves using a temporary framebuffer to create intermediate scenes, and then extracting pixel information from the framebuffer for blending. These requirements can be quite burdensome on a GPU-based graphics pipeline compared to current software rendering & texturing method.
• Sometimes, the GPU might have too many burdens to complete real-time game graphics.

On the other hand, directly outputting vector drawings without going through textures can be more efficient in terms of memory use & performance in situations such as:

• Super high resolution for Lottie Scenes.
• Outputting very simple Lottie design resources (such as icons) without any composition.

Obviously it could be more beneficial to unify the vector drawing primitives as a single form in the Godot, however, implementing complex rendering logic like Lottie directly in Godot can be costly in terms of development resources and maintenance. Therefore, using third-party libraries may be advisable. In such architectures, typically using FBOs or textures can provide a stable integration structure during the main rendering and thorvg rendering.

Moving forward, ThorVG would be able to attempt hardware acceleration through its own hw rendering backend in order to integrate with Godot's main rendering pipeline, targeting FBOs for drawing. Nevertheless, if performance issues arise, attempting an aggressive integration structure based on direct drawing methods and sharing rendering contexts might be considered, although it will be comparatively challenging.

[fire]

@beicause @hermet Is there a test project I can send to help ThorVG debug this?

[beicause]

@beicause @hermet Is there a test project I can send to help ThorVG debug this?

thorvg_test.zip

[hermet]

Hmm... it looks threading dead lock. let me review the patch first.

[hermet]

@beicause Could you please check the comments?

Also an opinion:
I just noticed that generating all frame images like this is quite intensive. I expect that dynamically updating the texture every frame by using ThorVG animation would be better for saving texture memory and reducing the harsh initialization bottleneck.

[beicause]

I think it's easier to use it in AnimationSprite2D/SpriteFrames by generating multiple frames at the same time. And if you just want one frame, you can set frame_count to 1 and set frame_begin. Playing the animation by dynamic updating the texture needs more codes for users.

Another option is implementing a import plugin to import Lottie as a image texture, like SVG. This avoids run-time cost, but increases app size.

[fire]

Another option is to treat this like theora like a video.

[beicause]

@hermet Hello, I made the change 6e56a6c according to your guidance, but still get stuck sometimes.

[hermet]

Thank you for your update. I have some additional feedback on your code, although it might not help to resolve this issue.

Unfortunately, I've tried similar calls with our ThorVG example with your attached resource(lottie.json in lottie_test.zip) using one extra thread but didn't encounter any issues, and even the thread sanitizer didn’t report anything.

I might need more context. Could you please conduct some tests and observe the results?

1. Disable the ThorVG threading feature and see the result:
   -> Remove #define THORVG_THREAD_SUPPORT in inc/config.h.
2. Does this worker thread seems not same with the thread which calls initialize_svg_module?
   
3. How many LottieTexture2D instances are created from this issue, and does update_image() calls are made just once per its instance?

[hermet]

.lot do you want to add this extension for lottie?

@fire That still seems to be in the early stages. ThorVG plans to support it in a future version update. (once the LAC announce it with update)

[Reuzi3]

I'm ok with https://docs.godotengine.org/en/stable/classes/class_image.html#class-image-method-load-svg-from-string style apis for lottie added.

I agree, and I think we need an API for Lottie. Making it more dynamic. Allowing to change/select States (frames) would also be great.

[beicause]

Although this PR is unable, I personally still have some reservations about it:

1. The JSON file must be renamed. Directly adding an importer plugin to the JSON file makes the JSON editor unusable. This might require an improvement in the editor's importer system to allow file editing when importing with Keep File (exported as is)
2. This PR merely converts Lottie to CompressedTexture2D. As an alternative, you could also use external tools to convert Lottie into images. So I wonder if this PR is necessary.

About runtime rendering of Lottie, I think it is not worthwhile for this PR. The performance of rendering Lottie in this PR is not good. Especially when rendering large images, it is very slow. Additionally, this increases the size of the export template (enabling Lottie increases the build size of the editor by about 200kb)

[fire]

We have more leeway to add lottie to the editor and less leeway to add kilobytes to the export template.

I want to do lottie integration [in] stages. Adding lottie for import conversion is a good stage 1.

Let's not add runtime rendering of Lottie yet.

The Godot Engine text editor does not function with a json the length of lottie. I think this is a separate bug.

[mbasaglia]

That still seems to be in the early stages. ThorVG plans to support it in a future version update. (once the LAC announce it with update)

It's been ratified by IANA https://www.iana.org/assignments/media-types/video/lottie+json

[fire]

Can we add the .lot extension now that it is IANA ratified?

[fire]

Needs a rebase on master because of thorvg changes.

Error: modules/svg/editor/resource_importer_lottie.cpp:93:17: error: no matching member function for call to 'load'
        res = picture->load(lottie_str.utf8(), lottie_str.utf8().size(), "lottie", true);
              ~~~~~~~~~^~~~
thirdparty/thorvg/inc/thorvg.h:1294:12: note: candidate function not viable: no known conversion from 'CharString' to 'const char *' for 1st argument
ranlib modules/libmodule_webrtc.macos.editor.x86_64.a
    Result load(const char* data, uint32_t size, const std::string& mimeType, bool copy = false) noexcept;
           ^
thirdparty/thorvg/inc/thorvg.h:1332:12: note: candidate function not viable: no known conversion from 'CharString' to 'uint32_t *' (aka 'unsigned int *') for 1st argument
    Result load(uint32_t* data, uint32_t w, uint32_t h, bool copy) noexcept;
           ^
thirdparty/thorvg/inc/thorvg.h:1272:27: note: candidate function not viable: requires at most 3 arguments, but 4 were provided
    TVG_DEPRECATED Result load(const char* data, uint32_t size, bool copy = false) noexcept;
                          ^
thirdparty/thorvg/inc/thorvg.h:1267:12: note: candidate function not viable: requires single argument 'path', but 4 arguments were provided
    Result load(const std::string& path) noexcept;
           ^

[beicause]

Can we add the .lot extension now that it is IANA ratified?

Done. I noticed thorvg also supports .lot extension thorvg/thorvg#3248

[fire]

Something is weird. I rebased on master and had some errors.

Error: modules/svg/editor/resource_importer_lottie.cpp:55:22: error: no member named 'parse_utf8' in 'String'
                err = manifest_str.parse_utf8(reinterpret_cast<const char *>(manifest_data.ptr()), manifest_data.size());
                      ~~~~~~~~~~~~ ^
Error: modules/svg/editor/resource_importer_lottie.cpp:73:20: error: no member named 'parse_utf8' in 'String'
                err = lottie_str.parse_utf8(reinterpret_cast<const char *>(lottie_data.ptr()), lottie_data.size());
                      ~~~~~~~~~~ ^

https://github.com/V-Sekai/godot/actions/runs/14289250169/job/40048338160

[beicause]

This PR needs to be updated. String::parser_utf8 has been replaced by String::append_utf8

[beicause]

Perhaps the doctool still need to modify to support inheriting resource importer. Currently the docs of texture importer part are all copied.

I noticed there are several PRs that might add texture import options. Lottie importer's docs need to update every time the texture import options change.

[beicause]

Handle ResourceImporterLottie individually in doctool now so that the docs doesn't include texture importer's options. This is a bit of hack, but still better than updating docs manually.

[beicause]

Converted to draft as this can make use of #105342 to reimplement.

[lyuma]

We should not make this dependent on #105342 to get merged, and even if you did want to wait for another PR, there is no need to mark this as a draft.

If you want this merged into Godot 4.5, I would encourage you to mark this ready for review, it can get merged, and then after #105342 is merged (if it is agreed on and merged), then another PR can adjust the Lottie (or SVG) importers to use that new texture importer system.

[beicause]

Alright, I mark it as ready and let teams decide whether to merge.

I removed the support for dotLottie as it's relatively incomplete.

[beicause]

We can't access metadata set by importer in script? In this case, #105342 won't be able to achieve the conversion to AnimationPlayer2D like this PR. After importing lottie as texture, the information about animation is lost, and CompressedTexture doesn't support to serialize metadata...

[beicause]

I removed the feature of dropping AnimationPlayer2D from this PR. I wonder if there is a more general approach to support it for sprite sheet textures.