Memory leak

[juanep97]

Repeated calls to the astrometry solver (hundreds) end up increasing the allocated memory until it overflow (currently circumvented it in my script by spawning a forked process for each field; which frees the allocated memory).

Versions:

• python 3.10.9
• astrometry 4.1.2

Steps to reproduce:

• Repeatedly solve a field hundreds of times (can be the same field, over a loop).

(thanks for the nice module, btw)

[alexw-im]

Just wanted to say thank you for catching this and mentioning it!

[juanep97]

I will add a test scenario, if you think it's useful; although I don't know if this will be a problem of astrometry or astrometry.net.

[alexw-im]

I've just noticed that it seems @rlancaste ran across maybe the same memory leaks when adapting Astrometry.net for StellarSolver: dstndstn/astrometry.net#183 (comment)

Perhaps @aMarcireau could compare notes to find and fix them if and when it's a priority!

[rlancaste]

Yes, there definitely were some memory leaks. I had to correct a bunch of them as best I could so that KStars did not have significant memory leaks when plate solving. I cannot guarantee that every change I made was the best solution, but I tried my best. I think I managed to fix all the biggest ones. There also were some issues I never resolved, but it got to the point where they were pretty small issues and did not significantly affect performance.

[alexw-im]

Yes, there definitely were some memory leaks. I had to correct a bunch of them as best I could so that KStars did not have significant memory leaks when plate solving.

Do you happen to know whether any of them got up-streamed into the original astrometry.net repo?

Aside - thank you for your work! I've only recently taken up any kind of active interest in astrophotography and the first thing I realized is that 99% of the barrier to entry is the effort in realizing which tools do what you're trying to accomplish and figuring out how to jam them all together. Stellar solver in Kstars really goes a long way on that front.

[rlancaste]

So I know at least one person was mentioning that they wanted to try to upstream some of my changes. I do not have the time to do that now. I am a full time Physics and Computer Science teacher. I had to the time to do StellarSolver during the pandemic, but now that we are back full time and I have a couple of other projects I have been working on, I haven't had time for as much open source coding.

[juanep97]

Sorry, I'm quite busy lately with academic life... I add a note here for when I have time to dive and check the origin of the memory leak. I realized (2) below also happens:

1. Memory leaks if the solver is invoked repeatedly many times, even if running for a short time. Memory is not freed.
2. Memory increases to ~tens of GB only in one run, if position hint separation is high (180 deg).

It seems to be related to the loading of index files, maybe memory is not released after loading them

[aMarcireau]

@juanep97 Apologies for the very late answer. I found and fixed a bunch of memory leaks in https://github.com/neuromorphicsystems/astrometry/releases/tag/4.2.1. Can you let me know if your issue persists?

[juanep97]

Sorry, I got busy with other stuff. I will try to test it and let you know. Thank you for the update!