[Feature Request] allow unicode_map to support more than 128 characters

[alinelena]

Feature Request Type

• Core functionality
• Add-on hardware support (eg. audio, RGB, OLED screen, etc.)
• Alteration (enhancement/optimization) of existing feature(s)
• New behavior

Description

at the moment unicode_map supports ony 128 characters... looks a lot but one can easily overun as it happened to me.

[yeroca]

The index type is a uint8_t, so that should allow up to 256 characters, unless I'm missing something.

[alinelena]

documentation is clear
https://docs.qmk.fm/features/unicode

Due to keycode size constraints, i and j can each only refer to one of the first 128 characters in your unicode_map. In other words, 0 ≤ i ≤ 127 and 0 ≤ j ≤ 127.

there is also a macro in the code that i fail to find now... that makes clear the restriction

[alinelena]

here it is

qmk_firmware/quantum/quantum_keycodes.h

Line 189 in a70288b

#define UP(i, j) (QK_UNICODEMAP_PAIR | ((i)&0x7F) | (((j)&0x7F) << 7)) // 127 max i and j

[yeroca]

I see. UP() has this limitation, but UM() doesn't. So for now, if you don't need the unicode to change when shift is held, you can use UM(().

[yeroca]

It occurred to me that as a local change (i.e. hack), you could get more pair code space by redefining the size of the UM space to be zero, and then use all of the 8000 to FFFF index space for the pairs, This gives 15 bits instead of the current 14, and then you can get essentially 7.5 bits for i and j, giving 181 easily-defined unicode characters instead of the current 128 using some simple arithmetic instead of the shifts and masks. The main change would be then something like this:

#define QK_UC_BASE 181
#define QK_UC_PAIR_MASK 0x7FFF
#define UP(i, j) (QK_UNICODEMAP_PAIR | (MIN(j, QK_UC_BASE - 1) * QK_UC_BASE + MIN(i, QK_UC_BASE - 1))) // 180 max i and j
#define QK_UNICODEMAP_PAIR_GET_UNSHIFTED_INDEX(kc) ((kc & QK_UC_PAIR_MASK) % QK_UC_BASE)
#define QK_UNICODEMAP_PAIR_GET_SHIFTED_INDEX(kc) ((kc & QK_UC_PAIR_MASK) / QK_UC_BASE)

These definitions in keycodes.h would need to be changed too:

./keycodes.h:    QK_UNICODEMAP                  = 0x8000,
./keycodes.h:    QK_UNICODEMAP_MAX              = 0xBFFF,
 
./keycodes.h:    QK_UNICODEMAP_PAIR             = 0xC000,
./keycodes.h:    QK_UNICODEMAP_PAIR_MAX         = 0xFFFF,

These become:

./keycodes.h:    QK_UNICODEMAP                  = 0x8000,

// Make the max lower than the base value, so that the range is empty
./keycodes.h:    QK_UNICODEMAP_MAX              = 0x7FFF,  
 
./keycodes.h:    QK_UNICODEMAP_PAIR             = 0x8000,
./keycodes.h:    QK_UNICODEMAP_PAIR_MAX         = 0xFFFF,

This will hurt performance a tiny bit, due to the need for the divide and mod operators, but shouldn't be horrible.

Of course if you need more than 181 unicode characters, this won't work for you either.

[alinelena]

that will be enough for me. I will test and report. thank you for your time!

[yeroca]

Let me know if you have problems with it. I haven't tried this.. just looked at the code.

[yeroca]

I fixed mismatched parens in the UP() definition.

[alinelena]

I can confirm that now is working... in the sense I have all my UC correctly displayed.
the offending keymap is here.

[yeroca]

Great! Good to hear.