A recent pytstone benchmark ?

[MordicusEtCubitus]

Hi,
I am updating the wiki https://wiki.python.org/moin/ParallelProcessing with Nuitka.

The about page says:

We currently get a 312% speedup in pystone, which is a good start. (source: Nuitka version 0.6.0 with Python 2.7.)

But Python 2.7 is no more in the race and Nuitka 0.6 is not the latest one.

Do you have a more recent benchmark ?

Could the speedup comment be updated with each new Nuitka release ?

Thanks in advance.

[kayhayen]

There is https://speedcenter.nuitka.net/ but it's not very concise.

[kayhayen]

I am actually writing the script right now to execute a benchmarking script with many Python versions, compiled and uncompiled, in order to be able to more easily update the data. This is also interesting, to find out if there are generational degradations, as the ones I tried to address with the 1.4 release.

[MordicusEtCubitus]

Thanks, will be glad to try it.
On my side, I've tried a very simple benchmark:

import time
start = time.time()
N= 10**8
while N > 0:
    N *= 2
    N /= 2
    N -= 1

stop = time.time()
print(stop -start)

This runs in 20.5 sec with python 3.10 on i9 12th gen and 7 sec compiled by nuitka 1.4.7
Thought compilation would have produce a better speed as pypy3 (3.8.13) is doing the same in 0.43 sec

I will be glad to understand why the gap is so high between pypy and nuitka.

Thanks in advance

Gaël

[kayhayen]

Ah, this is because the C type int is not yet used, watch that space for 1.7 or so to have it. I would assume it will then achieve C speed for that kind of code. Sorry if it disappoints you now.

[MordicusEtCubitus]

Ok, I understand it right now.
pypy is compiling specifically each function call with native C type when possible, but nuitka keeps python behavior using python int type which is slower but can manage higher numbers !

So, this looks not so bad finally.

Thanks !

[kayhayen]

Well, actually with the in-place, it can be very good, because it works on that one Python object, and does not allocate a new one at all, or so I would be hoping. But it may be the case that not all your 3 operations are done. I would have to check that, but it's not on the table right now.

But 300% for this is not too shaby yet. With objects it would be barely any I think. Could also be N > 0 that it's bad at. Not sure if the comparisons at all that specialized yet, probably not. What is there right now, and there have been -= issues with floats, where absurd improvements were achieved, is more a demonstration of feasibility, but it is not complete at all. Becoming complete is I guess on the plate, once Nuitka kind of pre-generates this alternating code, whee then of course, we can like look at this code, and go "we expect no Python level operations at all" for this code.

For now, I will work on 3.11, then the class scalability stuff, maybe even as far as producing "compiled object" types, for faster programs. I am however, once the loop type shape analysis works, and thinking of it, that is probably holding above code back right now from directly knowing the type shape of N, this thread will be resumed.

As for new numbers, they are coming. But unfortunately they are not very much different for up to 3.10, and relatively speaking, I do not see 3.11 as benchmarkable yet.

[kayhayen]

This is an issue for the website more than for Nuitka itself.

[kayhayen]

This is not the most important one, e.g. the #109 and #108 are more important, but we work on automatically generating the demos, we can just have one that produces reference values for use on the web site on various benchmarks too.