Add EEPROM and focus support to ColormapOverlay #1434

EvyBongers · 2024-07-12T06:51:48Z

Quick attempt to implement focus support in ColormapOverlay. I feel this should work, but I don't have my keyboard with me right now, so any debugging I'll have to do later.

EvyBongers · 2024-07-12T06:52:36Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

 }

+EventHandlerResult ColormapOverlay::onFocusEvent(const char *input) {
+  return ::LEDPaletteTheme.themeFocusEvent(input, PSTR("colormap.overlay"), map_base_, no_themes_);


colormap.overlay seems like the most logical focus command to me. Let me know what you think.

obra · 2024-07-29T20:27:02Z

@EvyBongers did you end up testing this on hardware?

EvyBongers · 2024-07-30T04:54:24Z

I completely forgot about this. I'll see about testing it today

EvyBongers · 2024-11-13T13:24:03Z

So, I finally got around to testing this. The command returns the palette indexes as expected, but setting the indexes doesn't yet work 🤔

EvyBongers · 2024-11-13T15:17:54Z

So, I dug a little deeper to see what I was missing. I thought I had a quick fix by calling on ::LEDPaletteTheme.themeFocusEvent(), but the internal logic of the plugin never actually used LEDPaletteTheme to store its data because it was never intended to work as a theme.
Looking at #1416, it seems to me that the third PR that we discussed there was never created, so unless we want so implement a read only version here, this effort is going to have to wait until EEPROM support is added to Colormap-Overlay.

The confusing part is that Colormap-Overlay (seemed to) need to call ::LEDPaletteTheme.reserveThemes() in order to work with the palette, even though it never uses any of the theme logic. It's been ages since I implemented that part and I don't remember why I needed to add that call to reserve themes nor whether it's actually needed for the plugin to work with the palette at all.

Closing this for now

EvyBongers · 2024-11-13T15:33:20Z

Reopened because EEPROM supports needs to be added anyway and EEPROM and focus support should land together.

EvyBongers · 2024-11-13T15:38:41Z

Just found a comment about this on Discord. Cross-posting to have all of this together.

#1434 will need quite a bit of work yet, I think. My first attempt to make focus work for colormap overlay settings simply passed through the handling of the focus event to ::LEDPaletteTheme.themeFocusEvent(). I had a proper look at that today and I'm pretty sure that isn't going to work, or at least it isn't sufficient. That's partly because (iirc) I needed to reserve at least one theme to get ::LEDPaletteTheme.lookupPaletteColor() to work. However, the plugin doesn't store any other data in a theme. That means that I will either have to find a way to have the plugin work with themes or I'm gonna have to implement a second or different means for handing the focus command

EvyBongers · 2025-03-04T21:42:34Z

@obra I'm continued worked on this and I have a feeling I'm getting close to having a first draft that can actually compile, but I just found that my idea of using -1 for keys that don't have an overlay color isn't actually supported by FocusSerial at this point. See my comments here: Colormap-Overlay.cpp#L117-L120

At first glance it looks like implementing this in FocusSerial is as easy adding this bit of code. Can I add that as part of this PR, or would you prefer having a separate PR for that?

  void read(int8_t &i8) {
    i8 = Runtime.serialPort().parseInt();
  }
  void read(int16_t &i16) {
    i16 = Runtime.serialPort().parseInt();
  }

obra · 2025-03-04T22:14:34Z

Just pull it out to a separate commit and I'm good :)

…

On Tue, Mar 4, 2025 at 1:42 PM Evy Bongers ***@***.***> wrote: @obra <https://github.com/obra> I'm continued worked on this and I have a feeling I'm getting close to having a first draft that can actually compile, but I just found that my idea of using -1 for keys that don't have an overlay color isn't actually supported by FocusSerial at this point. See my comments here: https://github.com/EvyBongers/Kaleidoscope/blob/8db281269e9bf9d2b1cb97c6162f5aa434e45050/plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp#L117-L120 At first glance it looks like implementing this in FocusSerial is as easy adding this bit of code. Can I add that as part of this PR, or would you prefer having a separate PR for that? void read(int8_t &i8) { i8 = Runtime.serialPort().parseInt(); } void read(int16_t &i16) { i16 = Runtime.serialPort().parseInt(); } — Reply to this email directly, view it on GitHub <#1434 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALC2AH75N23PY24JSKBID2SYM57AVCNFSM6AAAAABKYKWNE2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOJZGAYDKMRWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: EvyBongers]*EvyBongers* left a comment (keyboardio/Kaleidoscope#1434) <#1434 (comment)> @obra <https://github.com/obra> I'm continued worked on this and I have a feeling I'm getting close to having a first draft that can actually compile, but I just found that my idea of using -1 for keys that don't have an overlay color isn't actually supported by FocusSerial at this point. See my comments here: https://github.com/EvyBongers/Kaleidoscope/blob/8db281269e9bf9d2b1cb97c6162f5aa434e45050/plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp#L117-L120 At first glance it looks like implementing this in FocusSerial is as easy adding this bit of code. Can I add that as part of this PR, or would you prefer having a separate PR for that? void read(int8_t &i8) { i8 = Runtime.serialPort().parseInt(); } void read(int16_t &i16) { i16 = Runtime.serialPort().parseInt(); } — Reply to this email directly, view it on GitHub <#1434 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALC2AH75N23PY24JSKBID2SYM57AVCNFSM6AAAAABKYKWNE2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOJZGAYDKMRWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Signed-off-by: Evy Bongers <[email protected]>

…lay/Focus Signed-off-by: Evy Bongers <[email protected]>

Signed-off-by: Evy Bongers <[email protected]>

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

obra · 2025-03-11T19:22:37Z

Sure!

…

On Tue, Mar 11, 2025 at 12:02 PM Evy Bongers ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp <#1434 (comment)> : > + // It appears that a call to ::LEDPaletteTheme.reserveThemes() is needed + // because it's where palette_base_ gets initialized. Since this plugin (and + // possibly others) don't actually use themes, requesting memory for storing + // themes doesn't make much sense. Maybe initialisation of palette_base_ + // could be moved to a setup() method, though maybe the palette should be + // split from palette theme altogether? That's fine by me. Want a separate PR for that too? — Reply to this email directly, view it on GitHub <#1434 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALC2BIP2L773YAYSO2ZSL2T4XKVAVCNFSM6AAAAABKYKWNE2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDMNZVHAYTAOBWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers

Functionally it now works in that the numpad keys in the default firmware always light up, regardless of the assigned keys and that the overlay colors can be changed/disabled over focus.

One thing I noticed is that, if an effect is active on layer 0, that effect keeps playing on all keys not in the numpad. To work around this, I could change the default sketch to apply color index 22 (for example, 16-22 should be black once users upgrade), so that this doesn't happen.

However, that isn't the only way the sketch now differs from the Numpad plugin. My tests showed that, when changing from the numpad back to layer 0, the overlay colors persisted until the point that the active color mode applied a different color. That is:

With any mode that lights all leds, the leds follow the color mode logic and the colors applied by Colormap-Overlay don't persist.
with modes like stalker of chase, the red color persists until the effect hits those keys and changes the leds
with LEDOff the color of the overlay persists indefinitely

Please let me know what you think about this, as well as the comments I left on my TODOs

EvyBongers · 2025-03-12T14:01:34Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

 namespace plugin {
 uint16_t ColormapOverlay::map_base_;
+uint16_t ColormapOverlay::overlays_base_;
+uint8_t ColormapOverlay::max_overlays_ = 64;  // TODO(EvyBongers): figure this out. How determine a good maximum?


@obra do you have a way to determine a good value here, or is this pure guess work?

do we want to be setting it to layer_count? that might (or might not) need an accessor.

layer_count is far too few, as an overlay only applies to a single key. Of course we could set it to layer_count * Runtime.device().numKeys(), but if anyone is gonna need that number of overlays, they're better off using Colormap rather than ColormapOverlay.

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

EvyBongers · 2025-03-12T14:12:08Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

+      uint8_t layer_     = (i - key_index_) / Runtime.device().numKeys();
+
+      overlays_[overlay_count_] = Overlay(layer_, KeyAddr(key_index_), color_index_);
+      overlay_count_++;  // TODO(EvyBongers): how to make sure that we don't exceed max_overlays_


Is there any way to communicate over focus that an error occurred? I could just break the loop when the limit is reached, but I wonder if there are ways to report back why it happens. Of course, if the limit above were to be set to max_layers * numKeys this wouldn't happen, but that doesn't make any sense because if you're gonna set led colors on every key you should be using Colormap instead of Colormap-Overlay

there isn't currently. There should be.

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.h

Signed-off-by: Evy Bongers <[email protected]>

obra · 2025-04-11T16:08:45Z

Functionally it now works in that the numpad keys in the default firmware always light up, regardless of the assigned keys and that the overlay colors can be changed/disabled over focus.

One thing I noticed is that, if an effect is active on layer 0, that effect keeps playing on all keys not in the numpad. To work around this, I could change the default sketch to apply color index 22 (for example, 16-22 should be black once users upgrade), so that this doesn't happen.

I don't really feel strongly here.

However, that isn't the only way the sketch now differs from the Numpad plugin. My tests showed that, when changing from the numpad back to layer 0, the overlay colors persisted until the point that the active color mode applied a different color. That is:

This one, you should be able to fix by forcing the active led mode to do a full repaint, no?

With any mode that lights all leds, the leds follow the color mode logic and the colors applied by Colormap-Overlay don't persist.

with modes like stalker of chase, the red color persists until the effect hits those keys and changes the leds

with LEDOff the color of the overlay persists indefinitely

Please let me know what you think about this, as well as the comments I left on my TODOs

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers · 2025-04-13T13:07:19Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

+        // TODO(anyone): For now we just break the loop. Once focus implements
+        // support for reporting back errors, this should report why we break
+        // the loop


Left a TODO to follow up on this discussion: #1434 (comment)

EvyBongers · 2025-04-13T13:09:49Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.h

-  Overlay *overlays_;
-  uint8_t overlay_count_;
+  static uint16_t overlays_base_;
+  static const uint8_t max_overlays_ = 64;  // TODO(EvyBongers): figure this out. How determine a good maximum?


Following up on #1434 (comment):

layer_count is far too few, as an overlay only applies to a single key. We could set it to layer_count * Runtime.device().numKeys(), but if anyone is gonna need that number of overlays, they're better off using Colormap rather than ColormapOverlay.

Understood. I think that starting at a fixed # like 64 seems fine. if we find folks needing it increased, we can do that.

EvyBongers · 2025-04-13T13:22:19Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.h

        if (color_index_ >= 0 && color_index_ < ::LEDPaletteTheme.getPaletteSize() &&
            color_index_ != no_color_overlay) {
-          count++;
+          count++;  // TODO(EvyBongers): how to make sure that we don't exceed max_overlays_


Per your suggestion, I added a static_assert in the method above, but since I need to count the number if overlays here, it seems impossible to use that here. Any ideas on how I could validate that here? Or should I move the logic for finding all specified overlays to the macro below?

I don't have an obvious answer for you here. What happens if you just return when they've gone over the limit?

EvyBongers · 2025-04-13T13:24:15Z

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp

+    // When settings are invalid, better to clean it all up to prevent unwanted things from happening
+    ::EEPROMSettings.invalidate();


Re: #1434 (comment)

Is this the correct way to handle this?

I don't believe so. the seal is taken care of automatically. and isValid is a whole-system level thing.
This should be the right pattern

EventHandlerResult Keyclick::onSetup() { // Request storage space and load settings bool success = ::EEPROMSettings.requestSliceAndLoadData(&settings_base_, &settings_); // If settings weren't loaded successfully, use defaults if (!success) { settings_.enabled = true; settings_.version = 0x01; updateSettings(); }

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers · 2025-04-13T15:01:24Z

This one, you should be able to fix by forcing the active led mode to do a full repaint, no?

Just implemented this and verified that it works.

EvyBongers commented Jul 12, 2024

View reviewed changes

EvyBongers marked this pull request as draft November 13, 2024 14:35

EvyBongers closed this Nov 13, 2024

EvyBongers reopened this Nov 13, 2024

EvyBongers changed the title ~~Add focus support to ColormapOverlay~~ Add EEPROM and focus support to ColormapOverlay Nov 13, 2024

EvyBongers force-pushed the feat/ColormapOverlay/Focus branch from 41ea3cb to 912fc39 Compare November 13, 2024 16:12

EvyBongers added 4 commits March 9, 2025 13:02

Add TODOs for implemending EEPROM and focus support

747dcb3

Signed-off-by: Evy Bongers <[email protected]>

Implement focus reading colormap overlays

7d250a8

Signed-off-by: Evy Bongers <[email protected]>

Read color overlays from focus

3da5d29

Signed-off-by: Evy Bongers <[email protected]>

Add missing imports

1f860be

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers force-pushed the feat/ColormapOverlay/Focus branch from 8db2812 to 2a6ac62 Compare March 9, 2025 12:39

Fix variable references

ae4ee9c

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers force-pushed the feat/ColormapOverlay/Focus branch from 2a6ac62 to ae4ee9c Compare March 9, 2025 13:01

EvyBongers mentioned this pull request Mar 9, 2025

Add support for reading signed integers from focus #1479

Merged

EvyBongers added 5 commits March 11, 2025 18:03

Merge remote-tracking branch 'upstream/master' into feat/ColormapOver…

ff93006

…lay/Focus Signed-off-by: Evy Bongers <[email protected]>

Fix type error

42d0777

Signed-off-by: Evy Bongers <[email protected]>

Add missing include in example sketch

90692a6

Signed-off-by: Evy Bongers <[email protected]>

Add sketch files to example

ea611d0

Signed-off-by: Evy Bongers <[email protected]>

Update TODO with preliminary findings

22d8790

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers force-pushed the feat/ColormapOverlay/Focus branch from 5113820 to 22d8790 Compare March 11, 2025 17:46

EvyBongers commented Mar 11, 2025

View reviewed changes

plugins/Kaleidoscope-Colormap-Overlay/src/kaleidoscope/plugin/Colormap-Overlay.cpp Outdated Show resolved Hide resolved

Some initial work for EEPROM support

50fbd80

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers mentioned this pull request Mar 11, 2025

Split reserving palette and reserving themes #1482

Merged

Overlays should be static to work with EEPROM

3c3218c

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers marked this pull request as ready for review March 12, 2025 13:55

EvyBongers commented Mar 12, 2025

View reviewed changes

Merge branch 'master' into feat/ColormapOverlay/Focus

11a8e54

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers added 5 commits April 13, 2025 14:38

Be consistent in how int vars are incremented

ad4b5bc

Signed-off-by: Evy Bongers <[email protected]>

When EEPROM has invalid data, don't use it

9505c90

Signed-off-by: Evy Bongers <[email protected]>

Remember to increment the overlay counter

3151f4a

Signed-off-by: Evy Bongers <[email protected]>

Stop processing Focus data once max overlays is hit

ad05608

Signed-off-by: Evy Bongers <[email protected]>

Ensure sketches don't exceed max_overlays_

7000299

Signed-off-by: Evy Bongers <[email protected]>

EvyBongers commented Apr 13, 2025

View reviewed changes

Have ColormapOverlay force active led mode to refresh

5d7583e

Signed-off-by: Evy Bongers <[email protected]>

		// When settings are invalid, better to clean it all up to prevent unwanted things from happening
		::EEPROMSettings.invalidate();

Add EEPROM and focus support to ColormapOverlay #1434

Are you sure you want to change the base?

Add EEPROM and focus support to ColormapOverlay #1434

Uh oh!

Conversation

EvyBongers commented Jul 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

obra commented Jul 29, 2024

Uh oh!

EvyBongers commented Jul 30, 2024

Uh oh!

EvyBongers commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EvyBongers commented Nov 13, 2024

Uh oh!

EvyBongers commented Nov 13, 2024

Uh oh!

EvyBongers commented Nov 13, 2024

Uh oh!

EvyBongers commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

obra commented Mar 4, 2025 via email

Uh oh!

Uh oh!

obra commented Mar 11, 2025 via email

Uh oh!

EvyBongers left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

obra commented Apr 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EvyBongers commented Apr 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

EvyBongers commented Nov 13, 2024 •

edited

Loading

EvyBongers commented Mar 4, 2025 •

edited

Loading