import takes ~30 seconds

[Snawe]

Hi!
I just upgraded my application to python 3.12.

Doing import emoji there takes arround 30 seconds there. Doing the same on python 3.11 takes less than a second.

Any clue?

is reproducable with a simple 2 line script like

import emoji
print (emoji.is_emoji("done"))

Using Windows atm with emoji 2.9.0

Something similar was already reported here: #274

[cvzi]

I have no idea at the moment.

If you have the time, could you check two things:

Does it also happen with just the single import line?

import emoji

And maybe test some older versions of the module, to see if it is a new change that introduced this problem. For example these versions:

2.6.0
2.0.0
1.5.0

Do pip install emoji==2.6.0 to install a specific version.

---

If you are interested in trying it, I could create a test version for you that only has English language (or whatever languages you need) as suggested in the other issue. That would presumably reduce memory usage overall and reduce start-up time.

[Snawe]

Does it also happen with just the single import line?

yes

regarding emoji versions:

2.9.0 => 34 sec
2.6.0 => 24 sec
2.0.0 => 13 sec
1.5.0 => 10 sec

But I just found out something really strange... Like I already mentioned int the beginning, this only happens on python3.12. With python3.11 this does not happen.
What I found out now, it only happens if I run it via vs code. So if I hit F5 in vs code, I get the above times.
If I run it directly in the commandline, I still see that the time doubled (more emojis I think), but the times are:

2.9.0 => ~0.06 sec
2.0.0 => ~0.03 sec

I still think that it has something to do with the emoji library, since every other doesn't take ~500times longer, but yeah....

Just to complete the list, with python 3.11 via vs code:

2.9.0 => 0.03 sec
2.6.0 => 0.03 sec
2.0.0 => 0.03 sec
1.5.0 => 0.03 sec

From the commandline:
2.9.0 => 0.03 sec
2.6.0 => 0.03 sec
2.0.0 => 0.03 sec
1.5.0 => 0.03 sec

[cvzi]

Wow, thanks for the details!
My first guess would be that VS Code attaches a debugger or something similar and that somehow changed between Python 3.11 and 3.12

I will look into it.

[cvzi]

I can reproduce it with Python 3.12 on Windows 10 when running with F5 in VS Code. If you run it without the debugger CTRL+F5, it doesn't happen.
(Also I see no problems with Python 3.11)

VS Code runs a file like this, when you press F5:
cmd /C "C:\Python312\python.exe c:\Users\cuzi\.vscode\extensions\ms-python.python-2023.22.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher 64938 -- C:\Users\cuzi\Desktop\mytest.py "

I have create a issue at debugpy, maybe they know why this happens: microsoft/debugpy#1496

[cvzi]

The problem seems to be the large number of lines in the file data_dict.py (currently ~87k lines).

The dictionary in the file can be compressed into a single line:

EMOJI_DATA = {
        '\U0001F947': {
        'en': ':1st_place_medal:',
        'status':
        ...

-->

EMOJI_DATA={'\U0001F947':{'en':':1st_place_medal:','status: ....

resulting in a file with just 46 lines. With the compressed file, the debugging runs as fast as in Python 3.11

[TahirJalilov]

@cvzi may be it's time to think about separation of languages into different files, like it was before, what do you think?

[Snawe]

oh wow! Thank you very much for looking into it! :)

[cvzi]

@cvzi may be it's time to think about separation of languages into different files, like it was before, what do you think?

I agree.

Not sure it will help enough regarding this problem though, because the dictionary would still be huge. It takes 4 minutes on my computer at the moment. Even if it cuts the time to 10% it would still take 25 seconds, far too long.

Putting the dictionary into a single line is obviously really ugly. But it would be a quick fix.

I guess using a different file format, not Python code, could solve this problem with the debugging. For example storing the dictionary in a JSON file and the loading the JSON file when the module is imported.

[lsmith77]

I guess right now there is no work-around when using Python 3.12?

[lsmith77]

@cvzi based on 6fb1321 you are releasing a work-around?

[cvzi]

I guess so. I am not really happy with putting the dict into a single line, but there seems to be no other quick work around. And VS Code is one of the most used editors at the moment and already there seem to be about 2000/day downloads from Python 3.12 of this library (according to PyPi stats).

I have deployed that commit on my own apps, and it seems to work, i.e. in release environment, not debugging.

@lsmith77 any change you could test if it actually solves the problem with VS Code for you? It does solve it on my computer. You can install from my branch cvzi/one_line_dict like this: pip install https://github.com/cvzi/emoji/archive/one_line_dict.zip and then just create a python file with import emoji and run it in VS Code with debugging i.e. F5

[cvzi]

BTW for reference:
I also tried to put each "sub-dict" (each emoji) on a single line instead of everything in one line:

EMOJI_DATA = {
    '\U0001F947': {'en': ':1st_place_medal:','status': fully_qualified,'E': 3,'de': ':goldmedaille:','es': ':medalla_de_oro:','fr': ':médaille_d’or:','ja': ':金メダル:','ko': ':금메달:','pt': ':medalha_de_ouro:','it': ':medaglia_d’oro:','fa': ':مدال_طلا:','id': ':medali_emas:','zh': ':金牌:','ru': ':золотая_медаль:','tr': ':birincilik_madalyası:','ar': ':ميدالية_مركز_أول:'},
    '\U0001F948': {'en': ':2nd_place_medal:','status': fully_qualified,'E': 3,'de': ':silbermedaille:','es': ':medalla_de_plata:','fr': ':médaille_d’argent:','ja': ':銀メダル:','ko': ':은메달:','pt': ':medalha_de_prata:','it': ':medaglia_d’argento:','fa': ':مدال_نقره:','id': ':medali_perak:','zh': ':银牌:','ru': ':серебряная_медаль:','tr': ':ikincilik_madalyası:','ar': ':ميدالية_مركز_ثان:'},
    '\U0001F949': {'en': ':3rd_place_medal:','status': fully_qualified,'E': 3,'de': ':bronzemedaille:','es': ':medalla_de_bronce:','fr': ':médaille_de_bronze:','ja': ':銅メダル:','ko': ':동메달:','pt': ':medalha_de_bronze:','it': ':medaglia_di_bronzo:','fa': ':مدال_برنز:','id': ':medali_perunggu:','zh': ':铜牌:','ru': ':бронзовая_медаль:','tr': ':üçüncülük_madalyası:','ar': ':ميدالية_مركز_ثالث:'},
    ...

That reduces the import time (as expected) but it still takes too long, about 15 seconds on my computer.

[lsmith77]

sorry didn't get to it today will try to do it tomorrow morning