Why would file.close() freeze/crash pico using micropython?

[AngeloFatica]

Hello,

I have had a freezing issue for a while now and I have found it to be caused by file.close() function when writing to a .txt file in my program. It seems to work sometimes but a good 33% of the time it will hang itself up on that portion specifically. The file is only around 25 lines and it is just storing 25 different numbers for me, so it is not large at all. I have tried using the 'with open' method with the same result. Is there something I need to be aware of when trying to close the file? Perhaps give it more time after the last item is written to close it? Right now it writes all items with a 100ms sleep in between each and a 100ms sleep between the last write and the close() call. This seems to only happen with the pico, not any stm I have tried. Any help is appreciated, thanks.

Edit:
I am using a W5100S pico for this and have tried v1.19 to v1.22 micropython and the issue persists.

[mendenm]

Is it possible you have written files so many times, the built-in flash is dying? I have a PicoW I was using as a data logger, which ran perfectly for 6 months, and then started freezing up frequently. It would record data, and once a day it would be downloaded and erased. The write count on each block was probably a few hundred. I'm pretty sure I have killed the flash.

[robert-hh]

Is it possible you have written files so many times, the built-in flash is dying?

It needs more than a few hundred erase/write cycles to damage a flash block. The vendors usually allow 100_000 cycles. And in normal operation conditions it needs more. In a test I made, it needed 5 Million writes. And VFS has wear level protection. FAT is a different story, because it overwrites the same blocks very often.

[AngeloFatica]

I do not believe so, as one of these sensor devices will only ever see a max of maybe 100 file read writes in their lifetime to be honest. I

[AngeloFatica]

I also have tried changing it to a JSON file instead of text and it seems to be behaving similarly. Perhaps it is an issue with memory fragmentation? Would the close function need a specific amount of unfragmented memory to perform? I have had issues in the past with fragmentation, but my free memory as it stands is above 100KB still. This is very strange behavior.

[AngeloFatica]

stack: 1796 out of 7936
GC: total: 193664, used: 107184, free: 86480
No. of 1-blocks: 783, 2-blocks: 225, max blk sz: 1046, max free sz: 1002

This is my micropython.mem_info() print right before the freeze happens on file close. Is it an issue of something needing more than the max free block size? A fragmentation issue?

[AngeloFatica]

Update:

The issue is fixed by eliminating my threaded portion of the code altogether. I understand that _thread doesn't have the best track record with micropython, but I still am unsure as to why it would affect file system/flash at all if it is not using it or sharing any variables?

[Unchained-wp]

@AngeloFatica Thank you for the info. I might just change to C/C++. I used Micropython because it is easy to use and to learn a bit of python but my primary experience is with C++. That should give me a slight speed advantage too.

[bixb922]

The file system that Micropython uses is Littlefs2 https://github.com/littlefs-project/littlefs

AFAIK Littlefs does not know about multithreading. As long as you do all file operation on the same thread or use mutex it should be safe.

[AngeloFatica]

@Unchained-wp same story for me. Micropython is super helpful to develop on quickly for proof of concept but does have its drawbacks like you mentioned the speed. But absolutely an awesome programming language nonetheless.

@bixb922 I used the file I/O on the main core and it was still freezing on file closing calls. I never tried it with mutex though. Perhaps that could help. @Unchained-wp maybe you could give this a shot and see if it helps you out?

[Unchained-wp]

I only write to the file in one spot currently, don't even read it. I am actually using a mutex, the write follows immediately after I update the relevant variables if more than 5 minutes have passed since the previous write. The same mutex is also used when the webserver in the other thread reads the variables. I'll check the code again to see if I didn't perhaps missed using the mutex.

[MilhouseVH]

@AngeloFatica you may have experienced the issue discussed in #10448 - hopefully this is now fixed in 1.23.0, but if not then the solution (as you've discovered) is to avoid using both cores when using the WIZnet network module.

[peterhinch]

Re threading on RP2 you might want to read this doc.

tl;dr It needs using with great care.