Skip to content
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b529179
Initial draft.
Mar 14, 2019
ebe3189
Updates.
Jul 26, 2019
9033116
Consistant naming.
Jul 26, 2019
ee0d8ad
Rename back, tweak.
Sep 26, 2019
b7c68e3
Tweaks.
Oct 7, 2019
fd070c3
Some further verbiage.
Oct 7, 2019
92771ff
Add formal schema.
Oct 7, 2019
ca97835
Merge branch 'master' into ext/zell-fb-asset-variants
Oct 7, 2019
69b7edf
Merge remote-tracking branch 'upstream/master' into ext/zell-fb-asset…
Oct 7, 2019
8e5d8b1
Language tweaks, update Renee's contact info.
Oct 7, 2019
f0ab429
One more try.
Oct 7, 2019
a9bee21
Clarifications, expansion.
Oct 22, 2019
23915e9
Rename as khronos extension KHR_variant_materials
jercytryn Mar 9, 2020
07c109b
Rename extension KHR_variant_materials -> KHR_materials_variants
jercytryn Apr 14, 2020
46b16fc
Fix typos, non-normative disclaimers, and clarify tag order behavior …
jercytryn May 8, 2020
17fe9bd
Simplify extension to focus on wholesale static variant configuration…
jercytryn Jul 18, 2020
910746f
Fix example melded models
jercytryn Jul 24, 2020
b84ae8a
Clarifying note on no active variant
jercytryn Jul 31, 2020
fc9b8d9
Update README.md for alt text.
Aug 20, 2020
60be860
Apply 3D Commerce patch to schemas and spec.
Aug 21, 2020
09a22ce
Updated README, removed examples, and fixed mappings in spec along wi…
sleroux Aug 24, 2020
26d2629
Add new language defining mappings usage within viewer applications
jercytryn Aug 24, 2020
8543426
No relative paths and fix typo
jercytryn Aug 24, 2020
9bc4e4e
Bridge verb
jercytryn Aug 24, 2020
9049ea8
Change to top-down language to describe the specifics of the schema p…
jercytryn Aug 24, 2020
8af11ea
one more try
jercytryn Aug 26, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions extensions/2.0/Vendor/FB_material_variants/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# FB_material_variants

## Contributors

- Pär Winzell, Facebook, [@zellski](https://twitter.com/zellski)
- Renee Rashid, Facebook, [@debuggrl](https://github.com/debuggrl)

## Status

Draft

## Dependencies

Written against the glTF 2.0 spec.

## Overview

This extension allows for a compact glTF representation of multiple material variants of an asset, structured to allow low-latency switching at runtime.

A typical use case is digital commerce, where a user might be presented with e.g. a pair of sneakers and the ability to switch between different colours.

## Tagged Variants

We introduce a simple tag-based extension scheme, that allows for high-level runtime swapping between which `material` is used to shade a given `mesh primitive`: the extension root contains a mandatory `mapping` property, which is an array of objects, each one associating some set of tags with a material reference.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although in my customization work (athletic apparel and sports equipment) we are currently adding customization as a layer of logic around the model, I believe we should be able to replace a lot of the manipulation logic already with this declarative approach. Will try that out and report on the findings. But I have added some immediate conclusions below.


Imagine a sneaker with shoelace holes that are made from materials that depend on the overall shoe colour in non-obvious ways:

| Tags | Material |
| ---------------------------------- | ------------------------------- |
| `sneaker_yellow`, `sneaker_orange` | `shoelace_hole_material_brown` |
| `sneaker_red` | `shoelace_hole_material_purple` |
| `sneaker_black` | `shoelace_hole_material_yellow` |

(_the authors of this spec are not product designer, apologies for the dubious colour choices_)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you consider exposing in metadata all the choices that the users can make and all the values that they can specify? There might be more than just a single set of variants.
Those could drive the UI. And even if the UI is defined by the database this could be a way to ensure that the model is capable of representing all the same options exposed to the user.

If this ends up being applied, I would say that the tagnames would benefit from a well-defined format that combines both the property name together with the selected value. For example underscores might not be allowed for property and value ids and be reserved only for concatenating those together.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you consider exposing in metadata all the choices that the users can make and all the values that they can specify?

Yea I think we would need to expose multiple tag 'sets' through metadata. When we talk about driving UI with content from the glTF extension I get a bit wary as we should try to keep the data stored in the extension as business-logic free as possible.

If this ends up being applied, I would say that the tagnames would benefit from a well-defined format that combines both the property name together with the selected value.

I think having a clear and standardized way of tagging will make it easier for viewers to implement the spec so I can see this being useful regardless.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been brought up several times and consensus so far is to keep business-logic out of this extension. It can definitely lead to a rabbit hole of complexity as soon as there are thousands of combinations or rules start being needed (e.g The red laces can go with the yellow or red sole, but not the blue sole unless the purple heel is chosen).

It also puts a big burden on the 3D authoring tools. We can't expect 3D modelling tools to support defining the metadata of all possible configurations.


The currently active `material` for a `mesh primitive` is found by stepping through this array of mappings, and selecting the first one which contains any one of the currently active tag. If none match, fall back on vanilla glTF behaviour.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph suggests that an application would look for a material containing "at least one" of several currently active tags, whereas the example below (['sneaker_red', 'laces_gold']) suggests an application would be looking for all active tags, or at least some sort of "best fit".

I wonder if the tags should simply be specified as having some application-defined semantic meaning, and left at that. Discussion of how an application should navigate among its tagged materials could be left to a non-normative section, as I could easily imagine different applications doing that differently. Or do you think that would be too loose to have models using this extension be broadly portable?

Copy link

@mikkoh mikkoh Feb 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about tags more and my feeling is that tags don't represent the "first match" nature of this design well. In fact when I wrote a prototype application around this I had forgotten the spec. I actually implemented "best match" implementation where the material with the most matching tags was applied.

I feel like they should be called selectors or something denoting that one the first matching material will be applied.

Copy link
Contributor

@deltakosh deltakosh Nov 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does an app like Babylon.js Sandbox will know what is the active tag at load? Should we think about adding a "Default tag" information? Or do we consider that by default there is no selected tag so we pick the gltf vanilla material?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea that the default tag is the gltf vanilla material. No need to specify it. That way authors are able to specify what the default should look like even on viewers that don't implement this extension.


In other words, **this is not a literal mapping** in its glTF form – exporters, take note.

## The variant mapping as glTF JSON

A snippet of a mesh implementing shoe holes which uses this extension might look like:

```javascript
"meshes": [
{
"name": "shoelace_hole",
"primitives": [
{
"attributes": {
"POSITION": 0,
"NORMAL": 1,
},
"indices": 2,
"material": 0,
"extensions": {
"FB_material_variants" : {
"mapping": [
{
"tags": [ "sneaker_yellow", "sneaker_orange" ],
"material": 7,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you consider making adding visible as another property here and making all but tags optional?
In my work with customization we are frequently switching some small parts of the larger model based on user choice.
In terms of the example sneaker imagine that each of the variants was designed by a different designer and each or them have added a different volumetric brand mark at the back of the sole. Each of those logo geometries should be visible only when a particular tag is selected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if a visible flag is needed as the mapping is only used as a look up table that lists all of the materials that can be applied to the mesh. In the case of volumetric logos that have associated geometry, I'd say that would fall out of scope of the material variants extension as that would start changing the nodes a level up from the material.

},
{
"tags": [ "sneaker_red" ],
"material": 8,
},
{
"tags": [ "sneaker_black" ],
"material": 9,
},
],
}
}
},
// ... more primitives ...
]
},
// ... more meshes ...
]
```

The tag-based approach allows highly selective material switching, e.g. changing the colour of shoe laces only, or wholesale changes across diverse geometry, by reusing the same tags in switches across many distinct mesh primitives.

In the typical case, many tags will be active, e.g. `['sneaker_red', 'laces_gold']`.

Composition also works well with tags: if a scene were built from several different variational models, each with their own set of tags, then it's easy to imagine an API call on the entire scene that takes a set of tags and passes them on to each constituent model, recursively.

## Interaction with existing glTF functionality
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the intention is clear, but consider explicitly putting a non-normative note at the top of this section and the one below:

This section is non-normative.


The primary purpose of this extension is to simply formalise the idea of configurability, so that members of the ecosystem can meaningfully communicate: those who produce asset and those who ingest them, and all the tooling inbetween.

Beyond that, it's also a compression scheme: a way to compactly represent multiple assets that have a lot in common (e.g. most geometry, some textures). In this respect, it's most meaningful for the self-contained binary format GLB, where the asset file contains all its data up-front, both that which is shared between variants and that which differs.

If we also allow external references, i.e. images and buffers to source from URLs,then we can additionally come up with sophisticated mechanisms where geometry and textures are loaded separately, dynamically. There's pragmatic pros and cons to going down this route, all beyond the scope of this document. Each application will make its own decisions there; meawhile, the extension proposed works well regardless.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two paragraphs (Beyond that, ... and If we also...) seem to contradict one another, and may be confusing as-is. I think you could effectively condense them to something like:

As a secondary effect, material variants allow multiple assets — with shared geometry but different materials — to be stored more compactly. When using external URIs as references to textures, applications may (optionally) process geometry only once and lazily request texture assets only when needed for a particular variant.


On this section...

If we also allow external references, i.e. images and buffers to source from URLs,then we can additionally come up with sophisticated mechanisms where geometry and textures are loaded separately, dynamically.

... I would consider this a certainty rather than an "if"; incrementally loading parts of a glTF asset is already possible and supported in engines like three.js. It's probably out of scope for this document, as you say, but I would strongly expect (though not require) that applications will take advantage of it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestions here @donmccurdy, I've gone ahead and condensed the language of the non-normative sections as per your proposal

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we are presenting custom-made products we are generating a lot of textures based on user input (text, logo files, patterns, colors, settings, placements, etc.). In our customizers those are primarily generated on client side but in a generalized scenario those could be generated by a server if the server receives all the relevant properties for a given texture request.
In a more generic example this would be relevant for a custom dog collar on Amazon (I have ordered one and wish the presentation was more accurate).
Would you consider including something like that?

I recognize that there are concerns about wastefully requesting all textures based on updates of properties that do not impact the textures. We could remedy that, for example, by specifying which properties need to be passed to that or (in worst case) allow server-side redirects to links that contain only the needed properties.


## Implications for Applications and APIs

How does an application communicate to a glTF engine what the initial variant state should be? How does it submit a runtime request for a different configuration? It's out of scope for this extension to constrain or mandate an engine's public API, but a useful implementation will require something of the sort.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although we might not enforce users not making conflicting declarations I think we might need to cover the initial state by this spec in some way that helps avoid situations where the models by default look different on viewers that do not support this extension.
Maybe that could be solved by validation logic or a somewhat advanced json schema?

Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "FB_material_variants mesh primitive extension",
"type": "object",
"allOf": [
{
"$ref": "glTFProperty.schema.json"
}
],
"properties": {
"tags": {
"type": "array",
"description": "An array of variational tags.",
"items": {
"$ref": "glTFid.schema.json"
},
"minItems": 1,
"gltf_detailedDescription": "An array of variational tags, all of which map to the associated material."
},
"material": {
"allOf": [{ "$ref": "glTFid.schema.json" }],
"description": "The mapped material.",
"gltf_detailedDescription": "A reference to the material associated with the given array of tags."
},
"extensions": {},
"extras": {}
},
"required": ["tags", "material"]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "FB_material_variants mesh primitive extension",
"type": "object",
"allOf": [
{
"$ref": "glTFProperty.schema.json"
}
],
"properties": {
"mapping": {
"type": "array",
"description": "An array of mapping entries.",
"items": {
"$ref": "mapping.schema.json"
},
"minItems": 0,
"gltf_detailedDescription": "An array of mapping entries, each associating a set of tags with a material."
},
"extensions": {},
"extras": {}
},
"required": ["mapping"]
}