How to recursively remove files with mpremote?

[saraverbeecke]

I recently use mpremote instead of rshell and I like it, but I cannot find a good way to remove files from the device recursively (without supplying all the file paths manually). Has anyone any advice on how to do this with mpremote? There is a rm and a rmdir command but suppling -r argument is not supported and using rm /dir/*.mpy for example does not work...

[QiaoTuCodes]

you can look at here:

#!/usr/bin/python
# -*-coding:utf-8 -*-
"""
-------------- Description: ------------------
   FileName : main.py
   Author : isaac
   ProjectName : recursively remove files with boards.
   IDE Version : PyCharm
   Date:2020/11/1 14:20
   QQ:287000822 E-mail: [email protected]
------------------- END ----------------------
"""
__author__ = 'Isaac'

import os

m_list = []

m_list.append(os.listdir())

for i in m_list[0]:
    os.remove(str(i))
    print('remove all done!')

[jimmo]

I don't think that does what @saraverbeecke is asking -- this will non-recursively remove all files in a directory, but they're looking for recursive remove, as well as doing it from mpremote.

[jimmo]

This is a frequently requested feature for mpremote! It's on the todo list (PRs welcome!).

[ettore-galli]

Hi all,

I would like to share my personal solution to the problem.

I made this simple script that makes a full cleanup of all uploaded files. I then make a cleanup and a full upload of all files each time. If this is considered too drastic, the example can easily be tweaked to accomodate more sophisticated needs.

cleanup.py

import os

DIRECTORY_MARKER = 32768
FILE_MARKER = 16384


def notify(*args: str) -> None:
    print(*args)  # noqa: T201


def join(path_a: str, path_b: str) -> str:
    return f"{path_a}/{path_b}"


def list_files(base: str) -> list:
    # micropython: -> ignore attr-defined
    return [d[0] for d in os.ilistdir(base) if d[1] == FILE_MARKER]  # type: ignore[attr-defined]


def list_directories(base: str) -> list:
    # micropython: -> ignore attr-defined
    return [d[0] for d in os.ilistdir(base) if d[1] == DIRECTORY_MARKER]  # type: ignore[attr-defined]


def cleanup(base: str = ".") -> None:
    notify(f"Cleaning up {base}")

    for f in list_directories(base):
        file_to_remove = join(base, f)
        notify(f"removing file: {file_to_remove}")
        os.remove(file_to_remove)  # noqa:  PTH107 (micropython)

    for d in list_files(base):
        dir_to_remove = join(base, d)
        notify(f"removing dir: {dir_to_remove}")
        cleanup(dir_to_remove)
        os.rmdir(dir_to_remove)  # noqa:  PTH106 (micropython)


if __name__ == "__main__":
    cleanup(".")

Then, using mpremote, I execute the script the following way:

mpremote run path/to/cleanup.py

e.g. in my case:

mpremote run deploy/cleanup.py

Hope this can be useful to anyone...

[bulletmark]

You could consider using mpr. It merely wraps mpremote so doesn't offer recursive remove but you can "cheat" somewhat. E.g.

$ cd mydir
$ ls *.py
a.py  b.py  c.py
$ mpr rm // *.py
rm :/mydir/a.py
rm :/mydir/b.py
rm :/mydir/c.py
$

Of course this "trick" uses mpr's directory/path inference and exploits the wildcard file list generated by your shell to pass those local file names to the remote device. It assumes those same names exist in the analogous directory there.

[idcrook]

mpr definitely supports recursviely removing now, and has added to at least rm and rmdir

mpr rmdir --rf lib/micropython_xxx

[joewez]

Ha! That's a tricky question... I've been working on a simple on-board solution...

kill_ffs.py.txt

(save as 'kill_ffs.py' on the root path of the device)

It still has some issues with getting all the directories removed but you might be able to fix that... btw... it has code to skip the script itself and 'boot.py'. I think it would still work even if you still delete them.

Hope this helps!

[jimmo]

If all you want to do is nuke the filesystem, then just reinitialising the filesystem might be easier.

I do this a lot, so have an alias set up in mpremote to re-format the filesystem (with littlefs). See
https://github.com/jimmo/dotfiles/blob/master/common/home/.config/mpremote/config.py

(Unfortunately it's slightly different on each port, the stm32, esp32/esp8266, and rp2 versions are there).

[andypiper]

Late follow-up here, but I just needed this exact thing and your aliases are super useful @jimmo - thanks!

[peterhinch]

As a workround rshell has rm -r.

[Josverl]

Take a look at https://github.com/Josverl/mpremote_config/blob/master/snippets/wipe_folder.py

And the config.py a level above that As a way to simply add just about anything you can think of, as a command to mpremote.

[peterhinch]

Most of your scripts have invalid names: a - in a Python module name will produce a syntax error.

[Josverl]

I think that may be my Powershell verb-action legacy shining through...

Easier to fix on the filesystem than in the wetware ...

[peterhinch]

@Josverl Getting this to work has proved to be a very useful learning exercise re the extensibility of mpremote. Unfortunately your repo seems to be set up to disallow the raising of issues, hence my response here. The following are the steps I needed to take to get this to work. FWIW I use Linux Mint 20.3 and Python 3.8.10.

Firstly I cloned the repo into my home directory, with the result that config.py lives in /home/adminpete/mpremote_config. To get mpremote to find it I had to define an environment variable XDG_CONFIG_HOME. Please see mpremote source for the reasoning. However because of this line I had to move config.py and the snippets directories, with contents, to a new directory /home/adminpete/mpremote_config/mpremote.

I then hit a problem with this line in config.py because __file__ does not exist. I haven't researched this, but I think it's probably because of the way that mpremote executes the file. I used this revolting hack:

here = Path("/home/adminpete/mpremote_config/mpremote/config.py").parent.absolute()

whereupon it worked. At the very least I should interrogate the environment variable, but maybe there's a better way?

It would be nice to see these issues fixed, and some clear docs explaining how to install and use it, because it provides a very useful way to extend the capability of mpremote.

Getting back to the original post, the very useful wipe_folder works with the following syntax (which might not be self-evident to beginners)

$ mpr exec "folder='rats'" wipe_folder

however it leaves the folder intact (but empty). I would expect it also to remove the folder (easily fixed).

[Josverl]

The repo started as a way to replicate my config across different machines, so had only the bare necessities
I have added some info and enabled the issue tracker
Also I'll need to add a way to prevent inadvertently leaking secrets

The missing __file__ is due to a not yet merged PR that I already included in my temp mpremote fork at Josverl/mpremote
for the code see :
https://github.com/Josverl/mpremote/blob/291b219ebc753ec71b587dcbe9c591e382b86aef/tools/mpremote/mpremote/main.py#L355-L358
which is part of PR: #9573 ( yet unmerged)

Without that I also needed a ugly hack similar to yours, or copy the location logic from mpremote (not pretty either)

when i wrote the wipe_folder script I intended it to be run from the root. but agree is makes sense to try to delete the folder - but avoid erroring out if it is run from the root

[peterhinch]

I have now raised issues so we can stop hogging this thread :)

[robert-hh]

The rm() command of upysh.py in micropython-lib removes directories as well, if told so.

[shanisma]

(pip install mpr / https://github.com/bulletmark/mpr )
Why not just running: mpr rmd --rf /