Avatar Conversion

[Frooxius]

One of the major goals of this SDK is to allow easy avatar setup from the Unity Editor. Currently the SDK will convert scene "as is" by default, meaning that any skinned meshes are just part of the scene.

We need to introduce a workflow that will indicate that an object should be setup as an avatar and automatically convert it as such. There are a few possible ways to approach this for the user UI/UX:

• Add explicit "Convert Avatar" button to the UI - this will find avatar in the scene and convert it
• Add specific components to attach (avatar descriptor) which will be automatically picked up on conversion and make it into an avatar, without requiring specific button
  • This might be a bit more obscure to user, but those components will likely be necessary anyways, because some things need to be marked
  • Potentially this could be combined with the "Convert Avatar" button - if the user presses it and there's no marked avatar in the scene, they get a warning about this

From technical perspective, there are two main approaches:

1. Fully setup all the Resonite components through bindings on Unity side and then just transfer it as is
2. Utilize existing avatar setup routines - e.g. by calling a sync method (which would need to be exposed)
   • This needs additional ResoniteLink and Resonite changes, but would help keep the avatar setup relatively standard and reuse a lot of the existing code, rather than needing to reimplement this

There are other aspects to this too:

• We can potentially support realtime mode, but we'll likely need some adjustments - once the user starts wearing the avatar in-game, parts of it become driven - we should stop trying to update those

---

Are there any thoughts/wants on this so far? I want to collect some ideas before I start working on this bit.

[ko-tengu]

I think doing both options at once would be best, having the components and also a button to send a specific GameObject as a compiled avatar.

I also would prefer having a selector to choose a specific GameObject as the avatar root to create and send, since it's pretty common to have multiple avatars and versions in one scene and trying to intuit which specific version the user wants or demanding users to create a new scene with a single avatar are both problematic.

[Frooxius]

@ko-tengu Hmm... I think there's a few ways we could do that:

• Add a context menu option, so you right click the avatar you want to send
• Add a button to "AvatarRoot" descriptor that you click to run the conversion
• The SDK UI will scan the scene for all avatars and you pick from a list

[ko-tengu]

I like the UI and button in Avatar component options personally, though I'll also mention the option of the very Unity Standard drop box.

[Frooxius]

Yeah the drop box would be the picking from list option.

[troyBORG]

Trying to test out the avatar import and my shader conversion thing.

I made a new Unity project and imported the Zorrix Model Unity Project, then cloned the Unity SDK — had to run git lfs pull before the plugins actually worked. I'm also testing a conversion script for shaders. When I try to send the default scene that comes with the model (the one with "Zorrix With All Clothing Unpacked" in the hierarchy — T-pose fox with BodyMesh, HairSpikey_Mesh, ShirtMesh, etc.) I run into a few things that might be useful when you get to this:

RunDataModelOperationBatch comes back null for me even with Resonite running and Link connected, so nothing gets created on the Resonite side. Then the asset provider URL updates fail with "component not found" since those components were never created. Skinned mesh import also fails with that "Parameter 'index'" out of range error — I tried clamping bone indices and only sending bone data when bind poses are valid in my local copy but it still does the same thing, so it might be elsewhere in the pipeline that would need to be looked at..

Anyway, figured I'd mention it in case it helps when you're working on the avatar workflow.

[Frooxius]

@troyBORG Please hold off reporting any issues, the SDK isn't ready to use yet.

There's no point in reporting avatar issues until we take off the notice away from the repo and announce the features are ready - avatar conversion is not implemented yet and other dependent features aren't fully done yet and some of them require patches to the client aren't released.

TL;DR: Please do not report any issues until we say this is ready.