Patching of os not working properly when using it with behave

[schwaerz]

Describe the bug
Last two days I tried to get pyfakefs working with behave. Unfortunately I could not get it done. When I run the same code using plain python, the fakefs is working as expected. But when I run behave to execute the tests, calls to os.mkdir(), os.makedirs() or os.path.exists() will be executed on the actual file system. File I/O still will be done on the patched filesystem, though.

I am raising this issue here because I already tried the same thing using mockfs. Here patching of os.makedirs() and os.path.exists() works as expected, even when running using behave. Unfortunately mockfs does not provide all the features I need, therefore I'd love to use pyfakefs.

How To Reproduce
Use the small project attached, install requirements and run behave. It should fail to creating the directory as tried in environment.py.
behave-pyfakefs.tar.gz

There is also a foo.py, which does pretty much the same thing. python foo.py will work as expected.

Your environment

Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
pyfakefs 4.6.3

[schwaerz]

Some more info: Just realized that I could check the debugger:
Turns out that os is not replaced at all in this case. However - in the patcher - I will find the fake os module.

[schwaerz]

Looks like I found a "solution" in the meantime. Maybe it is even the way to do this anyways:

• In the behave test steps, os will not be replaced. Therefore I had to switch to using the patcher methods explicitly.
• In the productive code, os was replaced. Therefore overall I got it working now.

Not sure whether this is really as it is supposed to be?

[mrbean-bremen]

Interesting - at a first glance, this looks strange, and I have no idea what the problem is.
I haven't used behave, and I don't know what it does behind the scenes - maybe it does some patching itself that interferes with pyfakefs, but that is just guesswork.

I will have a closer look to understand the problem and will let you know if I find some way to fix this.

[mrbean-bremen]

Ok, as far as I understand, the problem is that the environment and steps modules do not appear in sys.modules, which pyfakefs uses to find all loaded modules and patch them. Behave seems to have some internal mechanism to load them, and it is probably not a good idea to undermine this by reloading them explicitely, as this might break the behavior.

There is one workaround I can think of: do not load the file system modules globally in the tests (e.g. in the environment and steps modules). You can change your example steps.py to:

from behave import given, when, then

@given("pyfakefs set up")
def step_impl(context):
    pass

@when('os.mkdir("/foo") is called')
def step_impl(context):
    import os
    os.mkdir("/foo")

@then('os.path.exists("/foo") will return True')
def step_impl(context):
    import os
    assert os.path.exists("/foo")

(the same goes for environment.py)
In this case the patching would work, because the modules are loaded dynamically, which is intercepted by pyfakefs.

This is not very nice, and I'm not sure if it is a practical way for your use case, but right now I can't think of anything else.
It may be possible to add some specific code for behave in pyfakefs, at the moment I don't see an easy way to do this though.

[mrbean-bremen]

I had a closer look, and indeed behave does not import the steps and the environment - instead it just loads the respective Python files, and directly compiles and executes them. I don't see a possibility to patch any functions with this workflow (at least without some major effort). Importing the steps breaks the behave tests, so this is not an option either.

Apart from the workaround above I don't see a way to make this work. I'll probably add this to the limitations sections of the documentation.

[schwaerz]

Thanks for the analysis. Probably I should raise a bug/feature request against behave?
Would be good to have this documented as a limitation 👍

Anyways, for me it will be sufficient to use the os functions in the productive code and only use the wrappers in the test code. That worked fine.

[mrbean-bremen]

Probably I should raise a bug/feature request against behave?

I think not - that would mean to change the implementation of behave sufficiently, and compatibility with pyfakefs is not enough of a rationale to do this. This is a limitation of pyfakefs, not of behave.

If you can work with the limitation, I'm inclined to close this issue, as both the limitation and the workaround are now documented in pyfakefs.