Skip to content
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 34 additions & 29 deletions proposals/2025-12-02_controller_shared_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,45 +76,47 @@ We will create a central object inside Mixxx that contains a triple-keyed map:
* Namespace (string)
* Entity (string)
* Key (string)
Comment on lines +77 to +78

@acolombier acolombier Dec 3, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of adding another dimension here, as opposite to stricly rely on the key, such that (["deck1", "button") would become deck1.button?
I am wondering if this second layer, which would likely induce the need for maps of maps, which could present more challenges in terms of stability and complexity (though performance is likely negligible since I doubt the scale we are talking about here is significant enough!)

Edit: just noticed the triple-keyed map, so I assume we would use a tuple instead, so I assume no impact on stability or complexity! Now just wondering about practicality?

@ywwg ywwg Dec 4, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no significant challenges in terms of stability, complexity, or practicality with the extra layer, and lots of benefits. Since the dominant use-case is hierarchical ("deck1","button"), I would prefer to encode that in the design rather than force controller mapping writers to invent their own methods to simulate that structure in a flat object. This also mirrors how we talk about control objects in Mixxx so it's consistent with the pattern we use everywhere in the codebase.

Furthermore on the read side, without the separation of objects every controller mapping would have to manually parse the key to determine which entity and object is being referenced. I prefer to lend them a hand and give them the structure we know will be used, no parsing required. (If they don't need an entity they can use "global" or whatever they want).

What issues of practicality do you see? That's kind of a vague word so I don't know what it's referring to

Managing the nested maps would all be handled in testable C++ code so I am not worried about juggling the maps and signals. The idea is to keep the complexity in C++ and make the javascript simple and predictable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What issues of practicality do you see? That's kind of a vague word so I don't know what it's referring to

It's just an additional grouping you need to do, which I don't see the added value for. IMO, having one arbitrary key is always better than two, especially when it would come to debugging/extending the mapping/code of someone else.

The idea is to keep the complexity in C++ and make the javascript simple and predictable.

This is a bit vague, could you clarify how this design make Javascript more "simple and predictable"?
From my perspective I see the opposite, where before you could have one wrong key (e.g problem with interpolation, typos, whatever...).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give some examples for your pro and cons arguments?

From my experience a hash single hash lookup is faster than three:

QVariant value = m_sharedValues.value(namespace + entity + key);  
QVariant value = m_sharedValues.value(namespace).value(entity).value(key);  

But that is an implantation detail that can be hidden.

The structured version allows nicely to iterated through an entity.
The first one would allow to add or remove layers dynamically.

But which requirements do we have for them?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea was to use a single hash with three key, much like your first line, but not sure.

The structured version allows nicely to iterated through an entity.

Would we want to allow an entity to be introspect-able? How would that look? Is this more from a debug/developer tool perfective or from a controller mapping side?
I guess the single key model would also allow that (much like the current developer tool) and could leverage "virtual" layer - a key such as deck1.samplers.slot1 could still be interpreted in a tree structure thanks to the ., without constrain on the number of layer, as opposite the entity/key design)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's keep in mind that these are very, very small maps (perhaps dozens of entries), nor do we need to worry about micro latency (buffer-size latency is probably the scale we should worry about: 1ms or more) so I would be shocked if speed of lookup is a concern.

I am looking at the mapping for the S4 and almost every single data update can be expressed as "entity:key:value" -- notably, the current mapping inverts this in a way that is inconsistent with most of mixxx, doing "key:entity" (padsmode:deck.group = value) rather than the more familiar (deck.group:padsmode = value) which is exactly the sort of wheel-reinvention I want to avoid.

I think it's easy to argue for fifteen different ways of designing this structure, which is why I'm very specifically choosing to be consistent with how Mixxx designed ControlObjects rather than try to invent something new.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I would be shocked if speed of lookup is a concern.

I agree that looking at the scale of a namespace, the hashing performance is probably anecdotal.

I am looking at the mapping for the S4 and almost every single data update can be expressed as "entity:key:value" -- notably, the current mapping inverts this in a way that is inconsistent with most of mixxx, doing "key:entity" (padsmode:deck.group = value) rather than the more familiar (deck.group:padsmode = value)

I totally agree that, for the current S4 implementation, entity:key works well, and it’s practical for what we have now. But as we look ahead—especially with plans for deeper screen interaction and library integration—this approach might introduce some ambiguity. For example, how would we handle the case like padsmode:deck.group? Would it be entity=padsmode, key=deck.group, or entity=deck.padsmode, key=group? If we don’t standardise this now, we risk creating inconsistencies that could confuse contributors down the line, but also, standard can often be perceive a barrier to entry/painful devx. IMO, this is why using a single flat key would be easier.

Now, there is a risk than once mapping uses padsmode.deck.group and another one deck.padsmode.key, but at least, this would be one single symbol (either a variable or a literal)

which is exactly the sort of wheel-reinvention I want to avoid.

Yes, I share the exact same interest here - I have the feeling that this "entity" concept tries to reinvent something that does not require reinventing.

As I have started working on library integration for screen, in light of the QML development, should I expect push backs when it comes to entity/key definition? For example, I currently have element such as library.table.sortingColumn - will I have complete freedom on how I see fit for what is my entity and key or could this create pointless PR reviews where one argues that it should be entity=library and key=table.sortingColumn, instead of entity=library.table and key=sortingColumn?

@ywwg ywwg Dec 16, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I think library integration needs its own proposal because I don't think controller scripts should be implementing library integration on their own. That is a huge task (just think of how complex our libraries are) and we do not want 50 implementations of a library widget for every controller.

Instead, Mixxx should provide a QML widget that controller scripts can plug in to their screens and customize for the available resolution as needed. The controller->screen communication would only be navigational controls: left/right/up/down/tab/enter/back etc.

As controller screens get more complex, we need to move more of the logic into mixxx, not into the controller scripts.

* Value ("SafeData")
* Value ("SafeValue")

#### Namespace

`Namespace` is a string that is unique to each controller **mapping
definition**. All connected controllers of the same model will share the same
namespace. e.g. Two CDJ-2000's will both have a namespace like `CDJ_2000`. All
hardware mappings must have distinct namespaces.
definition**. All hardware mapping configurations must specify distinct
namespaces.

Multiple device support is still out of scope for this proposal, but we
anticipate that this design can expand to support that use case. For example,
the namespace MAY have a suffix appended to distinguish distinct devices based
on an automatically-detected unique device identifier, e.g. two CDJ-2000's could
have namespaces like `CDJ_2000-ABCDEF` and `CDJ_2000-FEDBCA`. This suffix would
be applied in C++ code outside of the awareness of the controller mapping.

If a unique serial number cannot be determined at runtime, a special controller
preference (defined in Mixxx, not controller mappings) could be used to map
which controller is associated with which device through a new API, and this
value will be passed to controller mappings.
Comment on lines +87 to +97

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No change required, but just to highlight what was discussed in the meeting. The mapping developer will be able to define the namespace in two different way using the mapping manifest (XML):

  • A static namespace, common to all mapping instance, e.g CDJ-2000. In this case, all instances of mappings for any CDJ devices plugged will share the same namespace. This may look like something as
    <controller namespace="CDJ-2000">
      ..
    </controller>
  • A device-bound namespace - whether we use we the USB serial number (unlikely due to common clash existing across devices) or device handle (Bus X.Port Y or whatver form this takes a different OSes) or some other means is implementation details and not publicly exposed the namespace name to the mapping - in this case, we ensure proper the data is only shared across any instance of the mapping that could relate to this device, and failure to do so can be considered a bug, which we will intent to fix.
    As nice to have or future iteration, we will allow a user to change. This may look like something as
    <controller deviceBoundNamespace="true">
      ..
    </controller>

As a nice to have, or future iteration, we may allow the user to explicitly override the namespace, for example with a small dropdown/combobox/inputbox or whetever UX we decide on the controller setting, unrelated to controller settings.


#### Entity

`Entity` is a logical value defined by the controller mapping definition or
during the initialization of the mapping script (e.g. for readout of a serial
number using MIDI commands). It can be like a Mixxx-style group ("`[Channel1]`")
but it can be any arbitrary string. Many controllers will want to define
something like `deck1` to refer to a device that can itself be assigned to
multiple mixxx channels. For the case of multiple devices of the same type, it
is intended to implement functionality to gather a unique device identifier in a
later step. These will than be used as Entity to distinguish the identical
devices. These identifiers include, but are not limited to:

* USB device serial number (only works for USB and not each manufacturer use
unique serial numbers)
* Operating system provided device identifiers like Container-ID on Windows or
Location-ID on macOS

The controller mapping decides how these entities behave and Mixxx does no
`Entity` is a logical value defined by the controller mapping definition. It can
be like a Mixxx-style group ("`[Channel1]`") but it can be any of the
preselected names listed below. Many controllers will want to define something
like `deck1` to refer to a device that can itself be assigned to multiple mixxx
channels.

The controller mapping decides how these entities behave and Mixxx does not
enforcement of them. To reiterate: even if an "entity" "looks like" a Mixxx
group, it is not.
Comment thread
ywwg marked this conversation as resolved.

Here is partial suggestion for entity definition, which could be extended over
time.
Here is the proposed initial description for entity definition, which is
anticipated be extended over time.

```typescript
declare Entities {
type Mixer = 'mixer';
type Main = 'main';
type Library = 'library';
type Decks = 'deck1' | 'deck2' | 'deck3' | 'deck4';
type Decks = 'deck1' | 'deck2';
type Channels = 'channel1' | 'channel2' | 'channel3' | 'channel4';
type Controller = 'controller';
}
Expand All @@ -124,6 +126,9 @@ type Entity = Entities.Mixer | Entities.Main | Entities.Library | Entities.Decks
Numbered entity names could also be validated with a regular expression such as
`deck[0-9]+`.

'Entity' values are enforced by Typescript library code before function calls
are handed off to C++.

#### Key

`Key` is a logical value defined by the controller mapping definition. It could
Expand All @@ -142,7 +147,7 @@ implementation, we will support bool, number, and string, and Arrays of those:
```typescript
type SafePrimitive = string | number | boolean | null;

type SafeData =
type SafeValue =
| SafePrimitive
| SafePrimitive[];
```
Expand All @@ -158,22 +163,22 @@ The shift button on the left side of a Traktor S4MK3 would be stored this way:

pseudocode -- not final naming:

`m_shared_data["S4MK3"]["deck1"]["shift"] = true`
`m_shared_value["S4MK3"]["deck1"]["shift"] = true`

### API
Comment thread
ywwg marked this conversation as resolved.

The shared data API should be roughly the same across Controllers, QML, and C++.
The primary difference is that C++ will have access to the namespace value at
all times, whereas controllers and QML will have that value elided.

The controller javascript has access to the shared data object through three
The controller javascript has access to the shared value object through three
functions:

#### Get

excuse the pseudo-js:

`engine.getSharedValue(entity: Entity|string, key: string): SafeData?`
`engine.getSharedValue(entity: Entity, key: string): SafeData?`

`namespace` is set automatically by the engine code, so controllers can't get
that wrong.
Expand All @@ -182,7 +187,7 @@ This function returns error if the value is not found.

#### Set

`engine.setSharedData(entity: Entity, key: string, value: SafeData): void`
`engine.setSharedValue(entity: Entity, key: string, value: SafeData): void`

`namespace` is set automatically by the engine code, so controllers can't get
that wrong.
Expand All @@ -200,7 +205,7 @@ similar to how they subscribe to engine Control Object updates:

(not final naming)

`function makeSharedDataConnection(entity: Entity, name: string, callback: CoCallback): ScriptConnection | undefined;`
`function makeSharedValueConnection(entity: Entity, name: string, callback: CoCallback): ScriptConnection | undefined;`

In this first implementation, controllers can only subscribe to updates for
their own namespace.
Expand Down
Loading