Effects in OTIO

[jminor]

Currently OTIO has very limited support for effects. What other formats, specifications, schemas, and systems for interchange of audio & video effects exist out there?

OpenFX has come up a few times in conversation: http://openeffects.org/
Are there others we should research?

How do FCPXML, EDL and AAF handle audio/video effects?

[jminor]

We briefly met representatives of OpenFX today (hi!) and the pointed us to this set of open source OpenFX plugins which cover a wide range of standard effects: https://github.com/tuttleofx/TuttleOFX

This might help in our investigation & understanding of the OpenFX ecosystem, but also might be useful for offline rendering of an OTIO with effects into a video file.

I wonder, @darbyjohnston have you looked at OpenFX?

[darbyjohnston]

I haven't but I was hoping to take a closer look soon. The TuttleOFX project looks interesting though unfortunately the GitHub hasn't been updated for a few years and it looks like their domain isn't working.

[apetrynet]

I believe @fabiencastan is/was involved in TuttleOFX. He is also on the ASWF slack so still in the industry I guess :)

[fabiencastan]

Hello @apetrynet ,
Yes, I'm still in the industry and still involed in OSS.
I was deeply involved in TuttleOFX years ago, but the project is not maintained anymore as we did not managed to federate enough partners around it and the market has evolved. Also the project was completely based on OpenFX, which is a great standard but it is not updated regularly enough.
I would be happy to help if you have questions on openfx.

[apetrynet]

Thanks @fabiencastan!

[jminor]

FYI, I attended the OpenFX TSC Meeting today (2022-09-08) and we discussed a few ways that OTIO and OpenFX could collaborate.

1. Using OTIO to hand off OpenFX plugin info from app-to-app.
2. Time-interpolated plugin parameters. Currently OpenFX defers this to the host application.
3. Using OTIO to give an OpenFX plugin an in-memory view of the timeline it is operating in/on.

Clearly 1 and 2 overlap with OTIO's goals, and could lead to new OTIO schema for animated video effects parameters. 3 is an interesting potential use case for OTIO.

Perhaps a nice next step would be for someone to demo how OpenFX works to an OTIO audience, and to demo how OTIO's existing effects schema works to an OpenFX audience.

[jminor]

Also, for reference, here is the AAF Effects Protocol document, which describes how some effects are represented in AAF: https://www.amwa.tv/_files/ugd/f66d69_898ba231b4224b0280bb7ce9ce607a89.pdf

[markreidvfx]

If it helps, here is a collection of AAF files for all the effects in a stock version of media composer.
https://github.com/markreidvfx/aaf_filter_samples

[jminor]

Wow - there's a lot in there!

I wonder if there's a subset of these effects which are in both these AAF samples, and in these OpenFX example plugins which we could use to form a concrete use case.

[barretpj]

Serialising an OFX plugin's state in a host-independent manner should be pretty straightforward, it consists of

• a triple of {identifier string, version number, context} e.g. {"com.example.SuperEffects.Wipe", 3, Transition}
• the value of each parameter, keyed by name (the difficulty here is that any animation of parameters is entirely left up to the host application. So we'd need to define at least a way of serialising keyframe time and values and some indication of the interpolation mode)
• the upstream node (apologies if there's a better OTIO name here) which is connected to each of the plugin's named input clips, if this isn't already explicit in the surrounding OTIO structure

[ssteinbach]

If it requires serializing keyframe time/values, this would depend on:

#331

[Dithermaster]

I have thoughts on this. I've helped implement OpenFX in two hosts and associated first-party effects and I'm active in the OpenFX community. I'm passingly familiar with OpenTimelineIO.

OpenFX defines the host <-> plugin API. It does not specify any storage or persistence. However, some years ago I proposed an XML format for describing a plugin chain including the parameter values (static or over time). It was based on something we had done in one of our hosts for effect "presets" which could be applied in a project. It never moved forward within OpenFX, but I think an OpenFX / OpenTimelineIO collaboration would benefit both groups, and would define not only a way to include effects in an OpenTimelineIO project, but by using a subset / subtree of OpenTimelineIO it would enable a way to describe plugin chains that could be moved between host applications (even if they don't fully support OTIO, although nicer if they did).

I fully support such a project and am happy to contribute ideas and feedback. However, since I don't have a horse in the race right now I can't contribute specification or code.

In addition to timeline or project interchange between similar hosts, I think the real workflow value is bridging between dissimilar applications. A specific use case that comes to mind is communicating effect chains from on-set pre-visualization applications to post-production applications (like editing and compositing). So in addition to comparatively simple adjustments like ASC-CDL, higher-end effects could be confidently added to the pipeline such as lens diffusion simulation or beyond-CDL color adjustments (such as those popular in HDR workflows).

Note that AMPAS ACES has some efforts in this area in S-2019-001 ACES Metadata File (AMF), which is worth looking at as well.

[meshula]

Hello @Dithermaster! That sounds interesting. Perhaps it might be a topic to bring up during an OpenTimelineIO TSC meeting ~ everyone is welcome to attend and propose topics.

[jminor]

@Dithermaster thanks for the pointer to AMF. I had not seen that before.

We can look at AMF as reference for general effect support, and also maybe as a direct integration similar to recent discussions about Framing Decision Lists (FDL) which have similar workflow considerations.

Skimming the AMF spec, it looks like it is a superset of Color Decision Lists (CDL), and includes ASC_SOP Slope, Offset, and Power values, which we already have limited support for in OTIO. So this is a good point of reference.

@barretpj I would love to see an more detailed example of how an OpenFX plugin's state could be serialized. If we had a proof of concept, or mockup showing how a set of non-animated OpenFX parameters could be carried in an OTIO that would help to clarify many open questions. Tackling animated parameters could be folded in later.

[garyo]

Correct -- this is intended to be a mockup of an actual effect graph storage format, so not just a single OFX effect. I assume that's what OTIO is looking for.

As to your point about inputs, that's valid for the General (multi-input) context. For the OTIO folks: Generators have no inputs. In a Filter or Retimer context there's only one input, and in a Transition there are two, with implicit ordering (outgoing clip first). Paint also has two, with implicit ordering (source, mask). So perhaps the inputs for the General context should be a dictionary instead, keyed by the input name defined by the effect -- which as @barretpj says, can be an arbitrary string:

  inputs: {
    'first source': generator,
    'the mask!': other_effect
 }

Although I do think most (all?) hosts will present the inputs in order of definition, I don't think the spec requires that.

[barretpj]

Optional inputs mean there's ambiguity in a simple list, unless you have a way of specifying the unconnected state. I'd say relying on order of definition is weak, for example because a plugin could easily define clips in a different order to different hosts if it wanted to, maybe to make their UI look better in that host. All OFX clip access is done by name, not by index.
I'd also argue for using an inputs dictionary everywhere, not making the General context special.

[garyo]

I agree. Since General context is the only one where optional inputs are allowed, this spec should always require a dictionary for inputs in that context. Also, since the OpenFX spec mandates particular input names for the other contexts (e.g. SourceFrom and SourceTo for transitions) it would be OK with me to use a dictionary keyed by input name in all contexts (rather than an array), if that's simpler or more future-proof.

I assume OTIO will want this spec to work for non-OFX effects; if some other API used an array of inputs rather than names, they could just be called input-1, input-2 in this saved form (as one example).

[garyo]

Also, though not particularly relevant to this conversation, an OFX input (Clip) can have a Label which should be shown in the UI, and can be distinct from the name.

[jminor]

This is very helpful, and becoming more clear to me as you talk about the details. The distinction between Generators, Filters, and Transitions should map well to OTIO concepts. We already have the naïve notion of a generator as a media reference for a clip which could evolve to align with the OpenFX concept.

[jminor]

At today's OpenFX meeting we looked briefly at the OTIO effects metadata that DaVinci Resolve outputs. It uses an application-specific metadata block to store effects parameters, keyframes, etc. This is great to see and could be a starting point for a generalized official schema for effects.

Here's a screenshot showing an example:

[garyo]

One question is how different interpolation types should be represented. How far down that rabbit hole should OTIO go? One possibility would be just "Hold", "Linear", and "Spline" (cubic spline knots? interpolating points with optional in/out tangents?) Also ensure that keyframes are time-based, not frame-based (or at least make that choice explicitly). That gets complicated in a retimer context.

It would be good to have the inputs explicitly represented as in my example above. Also at least for OpenFX, context (generator, filter, transition, general) is important.

For OpenFX, please use the effect's kOfxParamPropScriptName since that is more stable than the user-visible name.

I think each effect will need some kind of unique ID so it can be referenced by downstream effects (not sure how OTIO handles graphs at the moment). See my example.

I would not suggest including "Default Parameter Value" -- but I would suggest always including all param values even if they are at the default. This is robust against the effect changing its defaults in a new version, breaking the comp.

[Dithermaster]

@jminor
What's the current status of this? The OpenFX group is happy to work on the plug-in effect side of this, such as coming up with a standardized serialization format for the plug-in chains (including parameters).

[jminor]

@Dithermaster FYI @jhodges10 is working on a big list/matrix of which effects we plan to support and how those map to various NLEs. I'll let him respond with details.

[darbyjohnston]

I have also been interested in effects with OpenTimelineIO and created a small open source project to experiment with some ideas:
https://github.com/darbyjohnston/toucan

Using OpenTimelineIO and OpenImageIO I wrote a small renderer that converts .otio files to image sequences. It currently supports multiple tracks, transitions, and about a dozen effects. The code is relatively small, only about 2500 lines of C++. I would like to add OpenFX support next, if anyone is interested in helping out it would be great to collaborate.

[garyo]

Hi folks; is there anything we in the OpenFX group can do to help this along? It would be great to have a standard way to represent effects in OTIO.

[jminor]

Hi @garyo - yes, the recent progress has been mostly from @darbyjohnston's work on adding OpenFX support to Toucan (see links above). I'll add some more notes from our recent discussion below.

[jminor]

We had some informal discussion about this in today's OTIO TSC meeting.

Here's a screenshot of an OTIO with effects that we looked at in the meeting. This OTIO came from an AAF, so the effect parameters are all in AAF-specific metadata. We can look at this as a reference point, similar to the Resolve example earlier in this thread.

% git clone https://github.com/OpenTimelineIO/otio-aaf-adapter.git
% uvx --from opentimelineio --with otio-aaf-adapter otioconvert -i ~/git/otio-aaf-adapter/tests/sample_data/keyframed_properties.aaf -o /tmp/keyframed_properties.otio
% raven /tmp/keyframed_properties.otio

[garyo]

Thanks for that -- it looks somewhat similar to my #921 (reply in thread). You might want to store the effect by its OpenFX name, usually something like com.darkstarsystems.plugins.myplugin, and it's probably best to store its version. You might also want to store the context (generator, filter, transition, general), maybe treating that as OpenFX-specific, i.e. in an OpenFX subtree of the metadata. The discussion in my reply above might be useful.
Don't forget vector-valued params like points and colors.
I assume OTIO has some way to represent which clips (or other effects, etc.) are connected to which inputs of the effect, i.e. describing the flow graph. In OpenFX inputs are known by their name, not their index.

[reinecke]

You might also want to store the context (generator, filter, transition, general)

Since OTIO has schema for GeneratorReference (something that yields synthetic media for a Clip), Transition, and Effect I wonder if which one of these it's encoded on is enough context to establish this?

[jminor]

In today's meeting, we discussed some next steps:

• @jhodges10 is looking into ways to publish the matrix of effect types that he gathered.
• It was suggested that we pick a small set of simple effects, without any animated keyframes, to support as a first step.

[darbyjohnston]

I assume OTIO has some way to represent which clips (or other effects, etc.) are connected to which inputs of the effect, i.e. describing the flow graph

Currently the effects in OTIO do not specify their inputs, I think the flow graph is implied from the timeline structure. Like a clip has a list of effects that are applied to it, and then all of the processed clips are composited together in track order. It's actually a bit more complicated since I think any timeline "item" can have a list of effects, but I think that's the general idea.

It does raise the question if the effects need to have explicit inputs to allow arbitrary graphs?

[garyo-gpsw]

Thanks. I guess that depends on whether you want graph-based VFX tools like Nuke, Flame, or Resolve/Fusion to be able to export OTIOs. If pure old-school pure editors are the only target, with effects on clips or transitions only, then you don't need a graph. But these days even AE/Premiere effects can have "clip inputs" as params, that are essentially pointers to other clips.

[jminor]

We should plan for both in the long term, but we could start with the simpler case.

I'm curious about the graph-based tools - do they require input nodes in the graph to exist somewhere on the timeline, or can they be completely off-timeline and only referenced in the compositing graph?

As a technical detail, we do have a mechanism (currently unused) within OTIO for references to other clips (look for OTIO_REF_ID in the C++ source).

[garyo]

Graph-based tools like Nuke don't have a timeline. They just render the graph, and clips are nodes in the graph like anything else. That might make it unsuitable for OTIO I guess.
But Resolve, for instance, is a hybrid. A "clip" on a timeline can be not just a clip, but a node graph describing the frames of that virtual clip. The default node graph is just in -> out. (I'm skipping lots of details -- check out videos on the "Fusion page" within Resolve.) Similarly in Flame.

[ssteinbach]

arguably a graph like nuke could be represented as a single track with a single clip that spans the duration of the nuke project timeline (IIRC nuke graphs have a master in and out point). The graph could be expressed as an effect on that single clip. I'm not sure exactly what the use cases are for exporting OTIO Of that kind though, I'd want to have some clear user stories or design intent around that use case.

[jminor]

In order to make some small first step on this, I propose that we pick a simple effect with only one input, like a 2D translate-rotate-scale (picture-in-picture), and look in detail at how it could be represented in OTIO's JSON schema.

Something like this:

What does that same effect look like in OpenFX, Resolve, Premiere, etc.?

[jminor]

Ok great. Here's an example closer to yours.

and an AAF, OTIO and MOV for reference.

Simple 2D PIP v2.zip

Can you export an OTIO of your example from Resolve?

[jminor]

Also, here's the Effect in OTIO as exported via AAF (snipped from the zip posted above)

{
    "OTIO_SCHEMA": "Effect.1",
    "metadata": {
        "AAF": {
            "ClassName": "OperationGroup",
            "ComponentAttributeList": {
                "_BORDER_FIXED": 0,
                "_EFFECT_PLUGIN_CLASS": "AVX2 Effect",
                "_EFFECT_PLUGIN_MANUFACTURER_ID": "3d780682-77e9-11d6-94d9-00306542a024",
                "_EFFECT_PLUGIN_MANUFACTURER_NAME": "Avid",
                "_EFFECT_PLUGIN_NAME": "3DWarp",
                "_EFFECT_PLUGIN_PRODUCT_ID": "1cf8c0c4-8536-45ab-aace-02abfeed5fdd",
                "_EFFECT_PLUGIN_PRODUCT_NAME": "",
                "_EFFECT_PLUGIN_SKU": "",
                "_EFFECT_PLUGIN_TYPE": "AVX2",
                "_EFFECT_PLUGIN_VERSION": "1.0",
                "_IGNORE_TRACKING": 0
            },
            "DataDefinition": {
                "ClassName": "DataDef",
                "Description": "Picture data",
                "Identification": "01030202-0100-0000-060e-2b3404010101",
                "Name": "Picture"
            },
            "InputSegments": {},
            "Length": 24,
            "MediaKind": "Picture",
            "Name": "OperationGroup",
            "Operation": {
                "Bypass": 1,
                "ClassName": "OperationDef",
                "Description": "",
                "Identification": "47052d96-effc-431a-b2fe-3b4291c5d8fd",
                "IsTimeWarp": false,
                "Name": "SBlend_v2",
                "NumberInputs": 2,
                "OperationCategory": "OperationCategory_Effect",
                "ParametersDefined": {}
            },
            "Parameters": {
                "AFX_FIXED_ASPECT_U": true,
                "AvidEffectID": "EFF2_SBLEND",
                "AvidParameterByteOrder": 18761,
                "DVE_BORDER_ENABLED_U": false,
                "DVE_DEFOCUS_MODE_U": "0",
                "DVE_FG_KEY_HIGH_SAT_U": "255",
                "DVE_MT_WARP_FOREGROUND_U": "1",
                "DVE_POS_ENABLED_U": true,
                "DVE_POS_X_U": {
                    "_aaf_keyframed_property": true,
                    "keyframe_baked_values": null,
                    "keyframe_interpolation": "Linear",
                    "keyframe_values": [
                        [
                            0.0,
                            -72.4465649
                        ]
                    ]
                },
                "DVE_POS_Y_U": {
                    "_aaf_keyframed_property": true,
                    "keyframe_baked_values": null,
                    "keyframe_interpolation": "Linear",
                    "keyframe_values": [
                        [
                            0.0,
                            -43.875
                        ]
                    ]
                },
                "DVE_POS_Z_U": {
                    "_aaf_keyframed_property": true,
                    "keyframe_baked_values": null,
                    "keyframe_interpolation": "Linear",
                    "keyframe_values": [
                        [
                            0.0,
                            0.0
                        ]
                    ]
                },
                "DVE_ROT_ENABLED_U": true,
                "DVE_ROT_Z_U": "37",
                "DVE_SCALE_ENABLED_U": true,
                "DVE_SCALE_X_U": "73",
                "DVE_SCALE_Y_U": "73",
                "DVE_TRACKING_POS_U": 0,
                "DVE_WARP_AMPLT_U": "0",
                "DVE_WARP_CURVE_U": "0",
                "DVE_WARP_FREQ_U": "59/100"
            }
        }
    },
    "name": "SBlend_v2",
    "effect_name": ""
}

[garyo]

I exported a .otio from Resolve, but looking at it I don't see the OpenFX effect at all. So that's probably not helpful.

Timeline 1.otio.zip

[jminor]

Interesting. I see some effects in there with no parameters, and one Dynamic Zoom with an animated dynamicZoomScale parameter.

"effects": [
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Transform",
                "Enabled": true,
                "Name": "Transform",
                "Parameters": [],
                "Type": 2
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Cropping",
                "Enabled": true,
                "Name": "Cropping",
                "Parameters": [],
                "Type": 3
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Dynamic Zoom",
                "Enabled": false,
                "Name": "Dynamic Zoom",
                "Parameters": [
                    {
                        "Default Parameter Value": [
                            0.0,
                            0.0
                        ],
                        "Key Frames": {
                            "0": {
                                "Value": [
                                    0.0,
                                    0.0
                                ],
                                "Variant Type": "POINTF"
                            },
                            "1000": {
                                "Value": [
                                    0.0,
                                    0.0
                                ],
                                "Variant Type": "POINTF"
                            }
                        },
                        "Parameter ID": "dynamicZoomCenter",
                        "Parameter Value": [
                            0.0,
                            0.0
                        ],
                        "Variant Type": "POINTF"
                    },
                    {
                        "Default Parameter Value": 1.0,
                        "Key Frames": {
                            "0": {
                                "Value": 0.8,
                                "Variant Type": "Double"
                            },
                            "1000": {
                                "Value": 1.0,
                                "Variant Type": "Double"
                            }
                        },
                        "Parameter ID": "dynamicZoomScale",
                        "Parameter Value": 1.0,
                        "Variant Type": "Double",
                        "maxValue": 100.0,
                        "minValue": 0.01
                    }
                ],
                "Type": 59
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Composite",
                "Enabled": true,
                "Name": "Composite",
                "Parameters": [],
                "Type": 1
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Lens Correction",
                "Enabled": true,
                "Name": "Lens Correction",
                "Parameters": [],
                "Type": 43
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Retime and Scaling",
                "Enabled": true,
                "Name": "Retime and Scaling",
                "Parameters": [],
                "Type": 22
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    },
    {
        "OTIO_SCHEMA": "Effect.1",
        "metadata": {
            "Resolve_OTIO": {
                "Effect Name": "Video Faders",
                "Enabled": true,
                "Name": "Video Faders",
                "Parameters": [],
                "Type": 36
            }
        },
        "name": "",
        "effect_name": "Resolve Effect"
    }
]

[garyo]

Yes, apparently Resolve applies some built-in Resolve Effects by default. But there is no Sapphire S_WarpTransform in there, which is what I used.

[jminor]

I'd also like to call attention to the Foundry's proposal for a 2D Transform schema here: #1794 (don't miss the linked video where they present the proposal and we talk about specific parameters).

[revisionfx]

"What does that same effect look like in OpenFX, Resolve, Premiere, etc.?" :
In Premiere transform is applied before effects - in AE transform is after effects - import Premiere project in AE does not even match within their own world. In Flame timeline if you have a resize parameter in an effect, you have to expand the buffer with Flame resize node before , in Resolve one has to work around the Auto-conform (what is the use case of saving Resolve native effects?), in FCP transforms can be concatenated across compounded clips. In practice I guess Nuke Studio would handle that... in compositing apps one can typically manipulate that as an effect rather than as a transform to a layer (not same thing).

OTIO should only be world of in and out point over a frame range to place a clip at a point into a timeline. Nothing else. Otherwise one might as well put a U in front of it and expect like software like that things only partially migrate. For example one might have an opentrack.io stream to time connect to a clip and an audio clip to time sync as well... You don't want a large file of camera related meta-data in the OTIO file, just external references. Or if you want that, you stripe this at the end with a clever way to seek for such items so the straight edit stuff does not become real slow to load... This sort of applies to effects too. [Also the industry is moving towards PTP synchronization which should be addressed by OTIO]. Note I have some reservations about 2D transforms I will explain in a separate item, as described by #1794 - limiting here to say framing an image (a DP like intent -or as Flex is intended in Avid) is distinct from animated in post a 2D window from an edit and then conform in another app perspective.

I am not clear about objectives here.
On top level is the idea of making OTIO an alternative-replacement for AAF? OK
Anything native effect is dark data in AAF parlance, is it a good idea to have the host save in OTIO effects parameters as opposed to just in OTIO a reference to some project file? If someone wants to make an app that collage a set of OpenFX plugins fine - OTIO should just have a reference to that app project file linked to some perhaps still virtual clip instead of a disk based render sequence URI. A special case of that for example in Resolve drx file now where the Color Page grading is saved and one can load it into another app. If one wants to change grading they go back to Resolve and save another drx file. That would be a more simple and practical example I think. Seems easier as a world model to have RPC dynamics (give me frame now or call owning app to pre-render this sequence as version changed) than trying to translate effects across hosts and API. From there at best Effects who works in multiple host types should be responsible on load OTIO and on save OTIO to write their own bytestream - so on load, have a way to match themselves and interpret in context (host.version) and set values themselves.

-pierre

[garyo]

Per the discussion the other day at the OTIO TSC meeting, here's a more fleshed-out example of a fictional LensFlare plugin with various parameters in an OTIO. I've removed all references to inputs & outputs, so this would be implicitly applied to its related clip. This is intended to be a high-level overview of a proposal; all elements are up for discussion.

Notes:

• I assume this will all be inside an existing "OTIO_SCHEMA: Effect.1" type, in the metadata block.
• Presence of an OpenFX block indicates an OpenFX effect.
• "context" can be filter (1 in, 1 out), generator (0 in, 1 out), transition (2 in, 1 out) or general (N in, 1 out -- maybe not appropriate for OTIO at this point).
• OpenFX effects all have well-defined unique names for each parameter.
• Param types demonstrated here: float, 2d float, color (RGB), boolean, and popup (or choice), represented as string -- see "lens".
  • Other param types are certainly possible. integer, string, filename etc.
• In this schema, if a param value is a JSON dict rather than int/float/string, it must have a "type" -- e.g. color, 2D XY point
  • I specifically used "2D XY" here for a resolution-scalable 2D float param. Other 2D float params might not be spatial, so would not change with resolution.
  • Colors could have a colorspace tag if desired. I'd suggest a default space of sRGB.
• I've shown a very simple animation format with keyframe values at times. Would need to be fleshed out (time coord sys: frames vs. seconds, more curve types, etc.)
• Per the OTIO TSC discussion, these blocks would be passed to a set of hooks; a com.darkstarsystems.lensflare hook could modify the param block during loading of this OTIO either to account for versioning or different source host app.
  • Such a hook could examine the actual clip resolution vs. the resolution at save time, to adjust 2D XY params in this example.

{
  "OTIO_SCHEMA": "Timeline.1",
  # ...
  "effects": [
    {
      "OTIO_SCHEMA": "Effect.1",
      "metadata": {
        "enabled": true,
        "OpenFX": {
          "Effect": {
            "ID": "com.acme-vfx.lensflare",
            "Name": "Acme VFX Lens Flare",
            "Version": "1.2.3",
            "Host": "Resolve",
            "Resolution": [1920, 1080],
            "Context": "filter"
          },
          "Parameters": {
            "lens": "105mm Prime",
            "scale_widths": 1.0,
            "rel_height": 1.0,
            "blur_flare": 0.0,
            "hotspot": {
              "type": "2D XY",
              "x": 1200,
              "y": 720
            },
            "pivot": {
              "type": "2D XY",
              "x": 960,
              "y": 540
            },
            "brightness": {
              "keyframed": true,
              "keyframe_interpolation": "Linear",
              "keyframe_values": [
                {"t": 0.0, "val": 1.0},
                {"t": 100.0, "val": 0.0}
              ]
            },
            "saturation": 1.0,
            "hue shift": 0.0,
            "rays_brightness": 1.0,
            "rays_rotate": 0.0,
            "rays_length": 0.3,
            "rays_thickness": 0.2,
            "hotspot_brightness": 1.0,
            "hotspot_color": {
              "type": "color",
              "red": 0.9,
              "green": 0.8,
              "blue": 0.2
            },
            "combine_with_bg": "Screen",
            "tint_background": true,
            "enable_motion_blur": true
          }
        }
      }
    }
  ]
  # ...
}

[revisionfx]

Who is responsible for writing this out, I would prefer it's the plugin or the plugin has option to flag let me do it- sync private data additional reason.

Different reasons:

1. Many of our plugins have tons of invisible parameters which are irrelevant in some hosts (sometimes parameters not even created if host A versus B). I prefer also just to save values changed - not all.
2. If we consider NLE outside of OFX hosts, it's likely easier if like say Premiere now who has some OTIO IO support that we could just write our own blob and interpret it on load. I tried with IO in Premiere of OTIO of our plugins and the net is a mess right now.
3. Not all API have matching parameter set. I realize we are OFX here.
4. Why is additional clip input not something we can address?
5. I see you use local time for KF, (t=0...) - just noting. Have to think.

[garyo]

In OTIO many times the plugin is not on the system when the OTIO is examined, translated, etc. In any case, OTIO will not want to (or even be able to) load and call into the plugin DLL.
The proposal is for there to be a hook (probably python) that a plugin could register with OTIO (at install time) to transform its param block from an OTIO data structure before being passed into the plugin instance so it can modify its param set and values. But remember that OTIOs are often created from AAFs or some other format -- the host will often not be writing OTIOs directly -- often a separate translation process. This is not a replacement for the host's native project format.
Additional clips: this was discussed at the meeting, and the OTIO group thinks storing arbitrary node graphs (N-in, 1-out) is a long-term goal but not currently planned. Filters and transitions are currently supported.
As for time, yes this is all TBD -- I presume OTIO has its own concept of time and its representation of keyframes in effects should respect it.

[revisionfx]

Avid and Premiere already have an export OTIO.
These hosts will export directly. They need to be enhanced so effects can be interchanged between hosts.
Will ask someone at Apple (FCPXML) as I am not clear what they do now with effects... and think about this. Might need to add to project assets a blob
Co-pilot says: "In applications like DaVinci Resolve or Premiere Pro, you can typically import FCPXML files to bring in the project and its associated data, including effects. " but I think it's lying to me again.
For interchange converter via Python that's a different issue. I can see cases where we might be needing to import settings saved somehow in the project bundle sort of things. Let me think.

Sorry I missed the OTIO meeting, I had some minor urgency to deal with