Mutations on arrays - why reconstruct array in memory every time?

[garygreen]

When adding an item to an array (e.g. a todo list, adding a new Todo) it's advertised to do this (using the spread operator):

  const [tasks, setTasks] = createSignal([])

  const addTask = function() {
    setTasks([...tasks(), { name: 'new task' }]);
  }

The problem with this is that it feels very wasteful - it's throwing away existing memory and then re-creating the array from scratch.

I'm new to solidjs so maybe it's my misunderstanding (I'm exploring switching from Vue, which doesn't re-create arrays you can simply push onto it, due to it being a Proxy).

I see there is a reconcile feature, but under the hood it looks like it's using https://github.com/solidjs/solid/blob/3d3207dd3aeb84c2a38377cf9f3b895995c2d969/packages/solid/store/src/modifiers.ts#L10 which re-creates the whole array in memory each time.

This isn't much of an issue with small arrays, but when dealing with super large arrays it's so wasteful to keep reconstructing that memory, garbage collection etc just to push one item onto it.

Is there a consideration for optimizing that in solidjs? Thank you for assistance! 😀

[mizulu]

from the docs
https://docs.solidjs.com/concepts/stores#appending-new-values

if you do append updates, you have a more optimal way to add new item
that should not go through the reconciliation logic

[garygreen]

So that's using updatePath but it's still using .concat and I can't see any reference to .push - concat in js will re-allocate memory for the array and rebuild it, rather than push which adds onto existing memory space, if available

[mizulu]

why would you want to see a push, where it is setting a key directly ?

also if you have a suggestion how the internals can be optimized, while still retain the same functionality
I think it will be appreciated, if you can create a PR with specific code changed, and show the performance improvements

[ryansolid]

The history of this came down to finding a single way to address arrays. The first problem with mutation is it will be the same object down stream which won't propagate. So helpers that don't know whether they are working on deep or shallow data it is safer to just clone. This doesn't apply to Store specific APIs, but things like mapArray or <For> component. Since that had no noticeable impact on benchmarks like JS Framework which deal up to 10000 rows I figured it isn't a big deal.

The second problem is that certain operations are a lot less performant when using proxies than others. Like yes you clone array, but if you unshift or shift or splice you hit the proxy trap on every index of the array after the point of change. So cloning an array looks like nothing compared to that. And if you want to add an item on the end you can still do that by directly setting the index.

@garygreen that part of updatePath probably could be more optimal. But that part of code is basically adding custom segments that pertain to part of the path and using it as a stack. Since children are mutating it I was concerned that it would interfere with other iterations. There is probably a way to write it so it doesn't need to do that. But since segments are being added dynamically thanks to helpers its more complicated than keeping fixed path and an index. And these situations are only hit when you are using special dynamic path syntax. Simple cases like above don't hit it.

... Actually the traversed path does.. hmm. That one might be simpler to optimize. Ironically I don't think anyone ever uses that feature.

We are moving away from the path-based setter as the default moving into V2. Since the API will be based on mutation it will fall into second problem but as you've witnessed with Vue etc it probably isn't that big of a deal. And you can always choose the most optimal operation if you are in the know. What I've witnessed is that these sort of performance issues are not a bottleneck. I even use the dynamic path segment in JS Framework Benchmark so never considered it an issue.

Moving to a discussion.