Skip to content

Abstract pool streams plug/unplug performance #329

Description

@Vladimirb87

I have a use case when one Kefir.pool has a lot of streams plugged. Current implementation of AbstractPool is very slow when try to plug/unplug streams one by one. Consider this as a example:

var pool = Kefir.pool();
var s = [];
for (var i = 0; i < 50000; i += 1) {
  var t = Kefir.constant(i);
  s.push(t);
  pool.plug(t);
}
var st = performance.now();
for (var i = 0; i < s.length; i += 1) {
  pool.unplug(s[i]);
}
s = [];
var et = performance.now();
console.log('Time: ', et - st);

Unplugging of all stream take ~8-10sec.

Have you considered changing the data structures used to maintain _queue and _curSources in AbstractPool? Currently, the arrays are cloned for each operation to ensure immutability. This cloning introduces an overhead of approximately 2x. I attempted to remove cloning, but it had to remain in _onActivation; otherwise, onEnd events might be lost in multiple stream combinations. With this approach, the tests pass.

However, the problem is that when _queue and _curSources are arrays, each stream removal operates with a time complexity of O(n).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions