Host 3D Support

[fxtech]

Open Effects Proposal for Standard Change

Please read the contribution guidelines first.

Standard Change Workflow

• Create proposal as issue (you're doing this now!)
• Tag this issue with standard change tag
• Identify subcommittee: at least one plug-in vendor, and at least one host
• Discuss the idea in this issue
• Write new or updated code and doc
• Publish updates as a pull request (ideally on a feature/PROPOSAL-NAME branch)
  • Make sure that PR references this issue number to keep them in sync
  • Discuss and review code in the PR
  • Meet all requirements below for accepting PR
• When subcommittee signs off and other members don't have any further review comments,
  maintainer merges PR to master which closes PR and issue

Requirements for accepting a standard change:

• Header files updated
• Documentation updated
• Release notes added
• Compatibility review completed
• Working code demonstrated with at least one host and one plugin
• At least two members sign off
• No further changes requested from membership

Summary

Some hosts have the concept of a 3D camera, lights, nulls, etc. There is currently no standard method
by an OFX plugin for querying that information.

Motivation

Plugin vendors want to get access to the host Camera and possibly lights and 3D nulls.

Problem

The problem is there is currently no standard way to query this information. Some hosts have
private suites but they are inconsistent.

Impact

This would require a new set of 3D-related properties and and a suite which would not affect
existing suites or properties.

Host support for 3D would be determined by the presence of a new suite.

Documentation Impact

A new section on 3D support would be required in the documentation.

Stakeholders

Hosts and plugins would benefit from this.

Discussion

The following hosts have private suites:

Nuke

Nuke has a NukeOfxCameraSuiteV1 with functions for defining a camera input (during the Describe) action that can be connected to via the graph, like an image input. Cameras are described as a PropertySetHandle with properties:

• Projection Mode
• Focal Length
• H/V Aperture
• Near/Far planes
• window translation/scale/roll
• Focal Point
• position matrix (a 4x4 matrix but its nature is not described)

Scratch

Scratch has a OfxAssimilateCameraSuite with a call to get a single global camera. The camera information is described by a PropertySetHandle with properties:

• Pose (a 4x4 OpenGL-style matrix),
• Field of View
• Near/Far planes.

HitFilm

HitFilm has comprehensive 3D support for Cameras, Lights, and arbitrary per-clip Transforms.
The Camera API is similar to Nuke's, where Cameras are treated like inputs. Properties are:

• Matrix (row-major 4x4 double)
• Zoom
• FOV
• Focus Distance
• Blur Level
• Aperture

Additionally, the suite allows querying the available lights which have numerous light-related properties.
Transformation matrices for clips that are "3D" can also be queried.

[fxtech]

It's worth pointing out the Alembic file format camera is defined by these properties:

• Focal Length
• H/V Aperture
• H/V Film Offset
• Lens Squeeze (aspect)
• Overscan (left/top/right/bottom)
• FStop
• Focus Distance
• Shutter Open/Close
• Near/Far planes
• Matrix (4x4 column major)

[revisionfx]

See also
https://openexr.com/en/latest/StandardAttributes.html
the new camera stuff is derived from SMPTE RIS (Virtual Production) attributes
There is a new initiative - https://www.opentrackio.org

[revisionfx]

Paul, you might want to ask Russ to look at this?

Also the folks from Maxon that were at the last meeting, and as well the person from Houdini...

This just in (Sam Richards, visual effects society ...) : on usd_camera slack
https://academysoftwarefdn.slack.com/files/U0191QYUGET/F084AJAAP3L/openlensio_lens_model_2.pdf

Pierre

[Dithermaster]

I know this is for virtual cameras in 3D software, but you should also look at how this is handled for real cameras, for the AR world.

One protocol is called FreeD. It is supported by Unreal Engine and many other software packages, as well as cameras from many companies. Here are some informative links:
https://ptzoptics.com/freed/
https://github.com/stvmyr/freeD

There is also the Stype protocol which supports more information, but has less vendors supporting:
https://support.disguise.one/s/article/Camera-tracking-protocol-FreeD-vs-Stype

It seems like you could interop between your virtual cameras and real ones by making sure your camera properties align with these.