node and ibm_db getting slower with each node release

[amcintyre99]

The subject says it all. And original odbc module has same issue.

timings
https://gist.github.com/amcintyre99/87ab958187f30989b816d238d341566d
code
https://gist.github.com/amcintyre99/4d13fbab65e78694accaf81ce95af6b9

db2 v11 on ubuntu 16.04 x64

any thoughts?

[bimalkjha]

@amcintyre99 If without changing the code of ibm_db, it performs slower with different versions of node.js, means issue is with node.js itself. Also, you already verified with two drivers. So, please open issue with Node. It does not seem an ibm_db issue. Thanks.

[bnoordhuis]

@bimalkjha Unless you or @amcintyre99 can point to specific reasons why it is slower, it is extremely unlikely that anyone at nodejs/node is going to look at it.

(I'm saying this with my node.js maintainer hat on. Not many people have the time or inclination to sift through thousands of lines of third-party C++ code on the off chance that the issue is with node.js and not the third-party module.)

[bimalkjha]

Well, in that case https://gist.github.com/amcintyre99/4d13fbab65e78694accaf81ce95af6b9 should be updated to distinguish between the time consumed by node.js code and C++ code. Thanks.

[amcintyre99]

If someone would tell me how to "distinguish between the time consumed by node.js code and C++ code", I will be glad to rerun the timings.

[bnoordhuis]

You can use perf(1) (see man perf) to get detailed traces:

$ perf record <flags> node --perf_basic_prof script.js
$ perf report --stdio

[amcintyre99]

this was fun
https://gist.github.com/amcintyre99/64bd41217a9e875c19a8f7f0c7fb7211

andrew@sherlock:/dma/odbc_ibm$ node -v
v6.10.3
andrew@sherlock:/dma/odbc_ibm$ uname -a
Linux sherlock 4.4.0-78-generic #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

i'm seeing strike through'ed lines above, not sure why

[amcintyre99]

also used the node profiler and got this
https://gist.github.com/amcintyre99/465514ff05ea8740d96162e2dc79f39b

here is the summary section followed by the first line of C++ section

[Summary]:
   ticks  total  nonlib   name
    291    0.8%    1.5%  JavaScript
  16269   43.4%   85.6%  C++
     84    0.2%    0.4%  GC
  18502   49.3%          Shared libraries
   2449    6.5%          Unaccounted

 [C++ entry points]:
   ticks    cpp   total   name
   5198   93.0%   13.9%  v8::internal::Heap::MoveElements(v8::internal::FixedArray*, int, int, int)

[bnoordhuis]

Your version of perf seems to be too old to understand the .map files that --perf_basic_prof produces but the profile from the builtin profiler is interesting: it shows that the main thread is mostly either doing nothing (blocked in syscall), busy mutating arrays (MoveElements) or waiting on a semaphore (semop).

Nothing really stands out but I noticed that lib/odbc.js is sloppy mode code. Adding a 'use strict' at the top might help, that gives V8 more leeway to optimize. The file needs to be strict mode compatible, of course, but I expect sloppy mode code will only get penalized more going forward so it pays to switch to strict mode.

By the way, is your nodejs binary from the ubuntu repos? Have you tried the official ones from https://nodejs.org/? I ask because some ticks are counted twice (once as C++, once as Shared libraries), which is a bug that doesn't exist in the official binaries as far as I'm aware.

[bnoordhuis]

I should add that since the lion's share is node sleeping in a syscall, most of the work probably takes place off-thread, in odbc's thread pool.

[amcintyre99]

re perf, am very current, probably just didn't point to the .map files
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-78-generic x86_64)

i will try 'use strict'

nodejs binary installed via latest nodesource package
https://github.com/nodesource/distributions

andrew@sherlock:~$ dpkg -l nodejs
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                           Version                      Architecture                 Description
+++-==============================================-============================-============================-==================================================================================================
ii  nodejs                                         6.10.3-1nodesource1~xenial1  amd64                        Node.js event-based server-side javascript engine
andrew@sherlock:~$ 

[amcintyre99]

with 'use strict' got this immediately

/home/andrew/dma/odbc_ibm/node_modules/ibm_db/lib/odbc.js:129
    delete db;
           ^^
SyntaxError: Delete of an unqualified identifier in strict mode.

i'm not much of a js coder so will need the package maintainers to try this change. thanks!!

[amcintyre99]

jeez, /proc/sys/kernel/kptr_restrict change required for perf

has symbols now
https://gist.github.com/amcintyre99/3a854a915c31b1548cfe68005a932a57

[amcintyre99]

both with symbols, they are very different
4.8.3
https://gist.github.com/amcintyre99/2ff3cb779fa404398e1a949fd30233d4
6.10.3
https://gist.github.com/amcintyre99/3a854a915c31b1548cfe68005a932a57