Path.cwd() different when used as default argument

[buhtz]

I'm not new to PyFakeFS but this took me a while to find out the cause of my problem. But I am still not sure if this is a bug or just a misunderstanding on my site.

When PyFakeFS is activated the return value of pathlib.Path.cwd() should be /.
But when I use pathlib.Path.cwd() as a default argument in a function it returns the working directory of the real file system.

See this code to reproduce:

#!/usr/bin/env python3
import pyfakefs.fake_filesystem_unittest as pyfakefs_ut
import pathlib
import logging
from logging.handlers import RotatingFileHandler


class Logging(pyfakefs_ut.TestCase):

    def setUp(self):
        self.setUpPyfakefs(allow_root_user=False)

    def test_foobar(self):
        foobar()


def foobar(dir_arg = pathlib.Path.cwd() / 'logs'):
    dir_local = pathlib.Path.cwd() / 'logs'

    print(f'dir_arg: {dir_arg}')
    print(f'dir_local: {dir_local}')

if __name__ == '__main__':
    print('Without PyFakeFS')
    foobar()

    print('\nPyFakeFS activated')
    test = Logging()
    test.setUp()
    test.test_foobar()

The output

Without PyFakeFS
dir_arg: /home/user/tab-cloud/_transfer/logs
dir_local: /home/user/tab-cloud/_transfer/logs

PyFakeFS activated
dir_arg: /home/user/tab-cloud/_transfer/logs
dir_local: /logs

But the output I would expect should be this

Without PyFakeFS
dir_arg: /home/user/tab-cloud/_transfer/logs
dir_local: /home/user/tab-cloud/_transfer/logs

PyFakeFS activated
dir_arg: /logs
dir_local: /logs

I'm using Python 3.9.2 on Debian 11 (stable) with pyfakefs.__version__ == '4.6.3'.

[buhtz]

Mhm... I thought I found my mistake. But using patch_default_args doesn't help either.

class Logging(pyfakefs_ut.TestCase):

    def setUp(self):
        self.setUpPyfakefs(
            allow_root_user=False,
            patch_default_args=True)

The output is the same.

[mrbean-bremen]

Thanks - this is indeed a shortcoming of the patching of default arguments. Apart from the fact that it is off by default (which you already found), it only checks the arguments for file system functions, not for file system classes (in this case pathlib.Path). This case is not mentioned in the documentation, so it is not obvious... It just never came up so far.

The only way to prevent this currently is to reload the module in question using modules_to_reload (provided that the module is not in the test module itself as in your example) instead of using patch_default_args=True.
I may check if it is possible to handle this case additionally if patch_default_args is set. This check is already slow, but as it is not used by default, making it slower yet should not matter that much. This may take some time, though...

[buhtz]

Thanks a lot. From my side you can close that Issue.

Just for documentation to others here the code that works. I separated it into two files.

Here is x.py:

#!/usr/bin/env python3
import pathlib


def foobar(dir_arg=pathlib.Path.cwd() / 'logs'):
    dir_local = pathlib.Path.cwd() / 'logs'

    print(f'dir_arg: {dir_arg}')
    print(f'dir_local: {dir_local}')


if __name__ == '__main__':
    print('Without PyFakeFS')
    foobar()

This is test_x.py

#!/usr/bin/env python3
import pyfakefs.fake_filesystem_unittest as pyfakefs_ut
import pathlib
import x

class Logging(pyfakefs_ut.TestCase):

    def setUp(self):
        print('PyFakeFS activated')
        self.setUpPyfakefs(
            allow_root_user=False,
            modules_to_reload=[x])

    def test_foobar(self):
        x.foobar()

[mrbean-bremen]

Thanks - I'll leave it open for a while to investigate if this can be fixed without the need to reload.

[buhtz]

I am sorry to bother you again. But maybe I created another edge case. The example code I used here worked with your solution. But porting that into my real productive code doesn't help. I assume this is because of the way I structure that package with it's sub-packages and how I import them.
But I don't get a solution.

Again the problem is that the pathlib.Path.cwd() used as defaut argument returns the CWD from real filesystem and not the fake one.

Please see the original repo buhtzology. The problem is located in the logging branch. Keep that in mind when cloning.

The relevant parts of the tree looking like this

buhtzology
├── README.md
├── src
│   ├── buhtzology
│   │   ├── _buhtzology.py
│   │   ├── __init__.py
│   │   └── ...
│   └── ...
└── tests
    ├── __init__.py
    ├── test_buhtzology.py

The code to test is located in the file src/buhtzology/_buhtzology.py (see the underline _). The functions in that file are imported in src/buhtzology/__init__.py.

The function to test is
https://codeberg.org/buhtz/buhtzology/src/commit/7be3a19dd3ad08d99f50eb6046c9d572a2d342ca/src/buhtzology/_buhtzology.py#L79

The test is here
https://codeberg.org/buhtz/buhtzology/src/commit/7be3a19dd3ad08d99f50eb6046c9d572a2d342ca/tests/test_buhtzology.py#L239

You have to install the package in Developer Mode via pip --editable . to make the tests work.

I invoke the test via python3 -m unittest tests.test_buhtzology.Logging.

Would be glad if you could take a look into it. Maybe I just have to modify the modules_to_reload part?

[mrbean-bremen]

Ok, I will probably have a look at it tomorrow.

[mrbean-bremen]

Ok, I had a look, and the reason why it does not work is a bit obscure: it's a problem of the reload order. As buhtzology loads _buhtzology in __init__, that one should be reloaded first, so using modules_to_reload=[buhtzology._buhtzology, buhtzology] (note the order) will fix the problem.

[buhtz]

Bähm! Thats it!
Thank you so much.

From a users perspective reading the documentation I would suggest to create a separate section with such edge cases. Maybe you can link to that edge cases in the regular documentation about modules_to_reload and patch_default_args.

[mrbean-bremen]

Yes, that's a good idea. In my experience, most people don't read the documentation, but there are at least some for whom that may be helpful 😀.

[mrbean-bremen]

After thinking about this I realized that there is really no way to fix this without reloading the code.
In principle, we could try to reload only the code used for the default args, which may be possible using some ast magic, but at the moment I don't want to go down that rabbit hole. I'll leave this as a limitation (now a bit better documented) and close this issue.