ISO SR : allow scan of subdirectories

[imakiro]

During our tests of the solution, it seems impossible to add an ISO storage repository and be able to retrieve ISOs located in the subfolders of this share.
It seems that this constraint exists since forever on xenserver...
What would it take to enable this functionality? is it possible? Was this requested and closed (if so, sorry, but I didn't find it using the "iso" keyword)

[olivierlambert]

I suppose this is the case to avoid very long SR rescan (going deep in the folder tree and filter all ISOs files inside).

I'm not 100% sure where the ISO filter code is, but it's very likely in Python so easy to hack and test. Would you like to make those experiments @imakiro ?

[imakiro]

Sure @olivierlambert , I'm all up for it and testing stuff is the least I can do to help right now.
Or maybe you could point me to the appropriate code segment?

[olivierlambert]

I suppose it's there: https://github.com/xapi-project/sm/blob/master/drivers/ISOSR.py

Current code parse all files and return only ISO and img files. So it would require extra code to inspect content of folders, probably with a depth of 1 folder to start.

[imakiro]

Ok, I'm having a look, but a quick question : under wich version of python is the project developped against?

[olivierlambert]

2.7 IIRC. @stormi ?

[stormi]

Indeed

[imakiro]

Ah, I've been hacking at the project with 3.8, I usually don't deal with obsolete versions of python and learned only Python 3 syntax.
It'll probably take a bit to setup a new environment with 2.7, I'll see what I can do.
update 1 : Ok, I need to also totally switch to a new distro : fnctl and syslog are totally linux dependant. Contributing to this project can't really be done easily...

[imakiro]

For this functionality, setting a depth parameter forbids the use of "os.walk", also is there a reason a non purely pythonic solution that runs an external command to list files / directories is used ?

[stormi]

We're not the original authors of that code so we don't know. Maybe the commit history of the repository can give a clue.

[imakiro]

succes !
Ok, so in util.py replacing
def listdir(path, quiet = False): cmd = ["ls", path, "-1", "--color=never"] try: text = pread2(cmd, quiet = quiet)[:-1] if len(text) == 0: return [] return text.split('\n') except CommandException, inst: if inst.code == errno.ENOENT: raise CommandException(errno.EIO, inst.cmd, inst.reason) else: raise CommandException(inst.code, inst.cmd, inst.reason)

with

def listdir(path, quiet = False, depth = 10): cmd = ["find", path, "-maxdepth", str(depth)] try: text = pread2(cmd, quiet = quiet)[:-1] if len(text) == 0: return [] return text.split('\n') except CommandException, inst: if inst.code == errno.ENOENT: raise CommandException(errno.EIO, inst.cmd, inst.reason) else: raise CommandException(inst.code, inst.cmd, inst.reason)

Does the trick !

[imakiro]

After having a whole lot of trouble with the project's signoff system, here's the pull request : xapi-project/sm#469

[TheFou]

Hi,
I see this has been open for a long time, without any implementation.
I'm not a not a developer unfortunately, but I really wonder how such a simple thing can stay stale for so long.

Also, one thing I don't understand : Why the need to scan all the isos in subfolders ?
Why not present the ISO SR "as is" with folders, maybe with a filter on some relevant file extensions, but nothing more ?
This way we can browse into subfolders as needed... especially since, if there are subfolders, it is probably for a proper management and sorting reason !
The current behaviour makes no sense.

Honestly, I find there are many little friction points like that, that make it really hard to go from VMware to XCP-NG.
I'm trying, but the itch to go back to VMware is really there.