Deno Fatal javascript OOM in CALL_AND_RETRY_LAST

[mokeyish]

It seems that this details has never been recycled.

[littledivy]

Does it still occur when you increase the old space size of heap? (using --v8-flags=--max-old-space-size=4096)

[mokeyish]

Does it still occur when you increase the old space size of heap? (using --v8-flags=--max-old-space-size=4096)

Here is my command:
deno run --v8-flags="--max-old-space-size=48384,--max-heap-size=16384" --allow-all ./fund-crawler.ts

[wperron]

Arrays are references to the actual value in memory; when you reallocate a new array to details it's generating garbage that the GC has to collect later on, that garbage accumulates over time and causes the v8 process to OOM. There's two things you can try:

1. Play around with the v8 flags to change the GC settings or the max allowed memory or,
2. Change your code slightly so that it handles reallocation slightly better

You could use details.splice(0, details.length) to empty the array in-place and avoid creating a new one. Depending on what the type of data is you could also use a more primitive type like a string and reset it to "" as that won't create a new object in memory. I'm sure there's things you could do with TextEncoder + Uint8Array too, or ArrayBuffer

[mokeyish]

Here is my code:

fund-crawler.ts.zip