problem with `os.stat` within `eyed3.load` when running multiple tests using `fs` based fixture

[stevenengland]

Describe the bug
Hello together, I have hard times finding out what causes the following error: I am using the fs (not fs_module etc. -> fs) inside a fixture. There is one test, that, when ran in isolation, succeeds. But when it runs together (sequentially) with any other test that uses the fs based fixture it fails.

TLDR
Of course the first thought is, are there any other filesystem actions colliding with the test but I think i ruled those out:

1: fs is used on a per test basis (as far as I can read from the docs only fs_module etc. are scoped broader than function, right?)

2: I added some debugging lines that verify the existence of the file which a code line later causes a fail. Also, before the failing line os.stat is called within eyed3.load method, eyed3 performs some internal checks like if not path.is_file() that are passed. So the pure existance of the file should be given I think. more can be read here: https://github.com/nicfit/eyeD3/blob/2cdef0d7542e65cc2756d63c2afffbdf48061eea/eyed3/mp3/__init__.py#L93

How To Reproduce
This is the test setup and test code up to the point where it raises:

# conftest.py
@pytest.fixture(scope="function")
def test_fs(fs):
    yield fs

# testmethod
    @e2e
    def test_copy_audio_with_fs(
        self,
        sut: AudioFileManager,
        test_assets_dir: str,
        test_fs,
    ):
        # GIVEN
        test_fs.add_real_directory(str(test_assets_dir))
        source_file_path = os.path.join(test_assets_dir, "0001.mp3")
        target_file_path = os.path.join(
            test_assets_dir,
            "target_repo",
            "9999.mp3",
        )
        os.makedirs(
            os.path.join(test_assets_dir, "target_repo"),
        )
        # WHEN
        sut.copy_audio( # <-- this line calls, besides others, the function _read_audio() that raises
            source_file_path,
            target_file_path,
        )

The following code is where it leaves my codebase and raises the error within eyed3:

    def _read_audio(self, file_path: str) -> eyed3.AudioFile:
        # some debugging lines ...
        print(f"file_path: {file_path}")
        print(f"files in parent dir: {os.listdir(os.path.dirname(file_path))}")
        print(f"is parent a dir: {os.path.isdir(os.path.dirname(file_path))}")
        print(f"is file_path a file: {os.path.isfile(file_path)}")
        print(f"size_bytes: {os.stat(file_path)[stat.ST_SIZE]}") # <-- this is the line used in eyed3.load internally that throws, see down below the stack trace
        audio = eyed3.load(file_path) # <-- this line throws when reaching the code line equal to the above one, see down below the stack trace
        # more stuff here that is irrelevant, because the line before fails
        return audio

The resulting debugging output is this:

file_path: [...]\tests\test_assets\target_repo\9999.mp3
files in parent dir: ['9999.mp3']
is parent a dir: True
is file_path a file: True
size_bytes: 3650

The error thrown/stacktrace is:

    sut.copy_audio(
[...]\src\mp3\audio_file_manager.py:52: in copy_audio
    audio = self._read_audio(target_path)
[...]\src\mp3\audio_file_manager.py:81: in _read_audio
    audio = eyed3.load(file_path)
[...]\.venv\Lib\site-packages\eyed3\core.py:452: in load
    return mp3.Mp3AudioFile(path, tag_version)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:146: in __init__
    super().__init__(path)
[...]\.venv\Lib\site-packages\eyed3\core.py:239: in __init__
    self._read()
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:164: in _read
    self._info = Mp3AudioInfo(file_obj, mp3_offset, self._tag)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:93: in __init__
    size_bytes = os.stat(file_obj.name)[stat.ST_SIZE]
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:1426: in wrapped
    return f(*args, **kwargs)
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:693: in stat
    return self.filesystem.stat(path, follow_symlinks)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:673: in stat
    file_object = self.resolve(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1760: in resolve
    return self.get_object_from_normpath(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1689: in get_object_from_normpath
    self.raise_os_error(errno.ENOENT, path)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:430: in raise_os_error
    raise OSError(err_no, message, filename)
E   FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '[...]\\tests\\test_assets\\target_repo\\9999.mp3'

Your environment
Windows-11-10.0.22000-SP0
Python 3.12.3 (tags/v3.12.3:f6650f9, Apr 9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)]
pyfakefs 5.6.0
pytest 8.2.1

does such a scenario ring a bell of someone? What can I do to further track down the issue?

Kind regards!

[mrbean-bremen]

Thanks for the report!
I cannot say out of the box what the problem is - it may be a problem with the package being incompatible with pyfakefs, a bug in pyfakefs or something else.
I will try to reproduce it given your information, though a self-contained reproducible example would be helpful, of course.

[mrbean-bremen]

If using your code with a dummy AudioFileManager that in copy_audio just calls _read_audio, I cannot reproduce the problem. Note that I have omitted the e2e decorator - I don't know it and how it works with pytest. I also did not see anything in the eyed3.load function that looks like it could not be patched, but I may have missed something.

I would need a bit more information to be able to reproduce this, ideally a self-contained example, as I wrote above.

[stevenengland]

Hi, I'll try to provide a minimal example tomorrow.

[stevenengland]

Hi @mrbean-bremen , here is a minimal example that raises for me:

@pytest.fixture(scope="function")
def test_fs(fs):
    yield fs

class TestFailingE2ETests:

    def test_read_audio_content_and_id3_tags(
        self,
        test_fs: FakeFilesystem,
    ):
        test_assets_dir: str = (
            "[...]\\tests\\test_assets"
        )

        test_fs.add_real_directory(test_assets_dir)
        audio = eyed3.load(os.path.join(test_assets_dir, "0002.mp3"))
        assert audio is not None

    def test_copy_audio_with_fs(
        self,
        test_fs: FakeFilesystem,
    ):
        # GIVEN
        test_assets_dir: str = (
            "[...]\\tests\\test_assets"
        )

        test_fs.add_real_directory(test_assets_dir)
        source_file_path = os.path.join(test_assets_dir, "0001.mp3")
        target_file_path = os.path.join(
            test_assets_dir,
            "target_repo",
            "9999.mp3",
        )
        test_fs.create_dir(
            os.path.join(test_assets_dir, "target_repo"),
        )

        shutil.copyfile(source_file_path, target_file_path)

        audio = eyed3.load(target_file_path)

        assert audio is not None

Each test running in isolation succeeds. But executing both tests in a sequence raises:

tests\test_audio_file_manager.py .F                                      [100%] <-- first test passes, second does not

================================== FAILURES ===================================
_________________ TestFailingE2ETests.test_copy_audio_with_fs _________________
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1750: in get_object_from_normpath
    target = target.get_entry(component)  # type: ignore
[...]\.venv\Lib\site-packages\pyfakefs\fake_file.py:566: in get_entry
    return self.entries[to_string(pathname_name)]
E   KeyError: 'target_repo'

During handling of the above exception, another exception occurred:
[...]\tests\test_audio_file_manager.py:235: in test_copy_audio_with_fs
    audio = eyed3.load(target_file_path)
[...]\.venv\Lib\site-packages\eyed3\core.py:452: in load
    return mp3.Mp3AudioFile(path, tag_version)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:146: in __init__
    super().__init__(path)
[...]\.venv\Lib\site-packages\eyed3\core.py:239: in __init__
    self._read()
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:164: in _read
    self._info = Mp3AudioInfo(file_obj, mp3_offset, self._tag)
[...]\.venv\Lib\site-packages\eyed3\mp3\__init__.py:93: in __init__
    size_bytes = os.stat(file_obj.name)[stat.ST_SIZE]
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:1456: in wrapped
    return f(*args, **kwargs)
[...]\.venv\Lib\site-packages\pyfakefs\fake_os.py:694: in stat
    return self.filesystem.stat(path, follow_symlinks)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:732: in stat
    file_object = self.resolve(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1834: in resolve
    return self.get_object_from_normpath(
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:1761: in get_object_from_normpath
    self.raise_os_error(errno.ENOENT, path)
[...]\.venv\Lib\site-packages\pyfakefs\fake_filesystem.py:489: in raise_os_error
    raise OSError(err_no, message, filename)
E   FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '[...]\\tests\\test_assets\\target_repo\\9999.mp3'
=========================== short test summary info ===========================
FAILED tests/test_audio_file_manager.py::TestFailingE2ETests::test_copy_audio_with_fs
========================= 1 failed, 1 passed in 0.66s =========================
Finished running tests!

As you can see it complains E KeyError: 'target_repo'. But I don't have a clue why.

[mrbean-bremen]

Thanks a lot, I now can reproduce the problem! Still a bit at a loss as to the cause, but I will see what I can find.

[stevenengland]

Thanks a lot in advance! And good luck :-)

[mrbean-bremen]

I'm quite sure that this is a bug in pyfakefs regarding the patching of pathlib by the dynamic patcher, but it may take some time to understand and fix it. For the time being you could try to switch off the dynamic patcher. You can use:

@pytest.fixture
def test_fs():
    with Patcher(use_dynamic_patch=False) as patcher:
        yield patcher.fs

as your fixture. This will fix the minimal test, but the dynamic patcher may still be needed for other tests, so this may or may not help you.

[mrbean-bremen]

I kind of got stuck with this issue, as I could not find an automatic fix for the problem, and I'm not sure if I will find one - I currently consider this a limitation of pyfakefs.
There is a workaround though that will fix this kind of issue by reloading the problematic module in your fixture:

def reload_cleanup_handler(name):
    if name in sys.modules:
        importlib.reload(sys.modules[name])
    return True

@pytest.fixture
def test_fs():
    with Patcher() as patcher:
        patcher.cleanup_handlers["eyed3.mimetype"] = reload_cleanup_handler
        yield patcher.fs

I've also made a PR (now merged) that adds this specific reload_cleanup_handler as a convenience function and fixes the documentation for the cleanup handlers in the troubleshouting chapter.

Note that in an earlier version I had done this reload to all dynamically loaded modules, but that turned out not to be a good idea because of certain unpredictable side effects in some modules.
I will think about this some more - maybe I'll find a way to detect which modules need to be reloaded, but for now, setting the cleanup handler is the way to go.

[stevenengland]

Thank you very much for your efforts. I'll give your convenience function a try this week.

Thanks

[mrbean-bremen]

FYI: I just made a patch release which contains these changes (among a couple of other fixes).

[stevenengland]

Hi @mrbean-bremen , I just updated pyfakefs to the latest patched version but ... unfortunately I am not quite sure what I need to implement to make it work. I think I need to follow your advice for dynamic patcher with patching a module (https://pytest-pyfakefs.readthedocs.io/en/latest/troubleshooting.html#tests-failing-after-a-test-using-pyfakefs). Returning to the minimal example with eyed3: Is the module my module that uses eyed3 or is it eyed3 or something within eyed3?

Thanks!