ImportError due to tests in editable installed packages

[starsirius]

Hey, guys. I couldn't find a good place to ask questions, so decided to post it here. If it's inappropriate, please advise and I will move it to somewhere else! Thanks!

I switched my mongnoengine installation to editable (along with nose2) to try out some features, and then the first test file (only the first one) of my tests starts to fail with an ImportError and the trace looks like nose2 was trying to run mongoengine's own test suites. I have tried a few things but still had no luck and no clue about how to address this issue. Any ideas would be super appreciated. Thanks!

My requirements.txt looks like this:

...
-e git+git://github.com/starsirius/mongoengine.git#egg=mongoengine
-e git+git://github.com/starsirius/nose2.git#egg=nose2

My nose2 config file:

[unittest]
start-dir = tests
test-file-pattern = *_tests.py
plugins = nose2.plugins.layers

And the traceback:

======================================================================
ERROR: tests.api.client_apps_tests (nose2.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.api.client_apps_tests
Traceback (most recent call last):
  File "/Users/starsirius/Code/project/venv/src/nose2/nose2/plugins/loader/discovery.py", line 201, in _find_tests_in_file
    module = util.module_from_name(module_name)
  File "/Users/starsirius/Code/project/venv/src/nose2/nose2/util.py", line 81, in module_from_name
    __import__(name)
  File "/Users/starsirius/Code/project/venv/src/mongoengine/tests/__init__.py", line 2, in <module>
    from document import *
  File "/Users/starsirius/Code/project/venv/src/mongoengine/tests/document/__init__.py", line 8, in <module>
    from indexes import *
  File "/Users/starsirius/Code/project/venv/src/mongoengine/tests/document/indexes.py", line 9, in <module>
    from nose.plugins.skip import SkipTest
ImportError: No module named nose.plugins.skip

Thanks a lot!

[aloysbaillet]

It looks like nose2 is correctly discovering tests within the "mongoengine/tests" folder, and then trying to load the document.indexes module, which in turn is trying to import nose1's skip plugin, which is not found, as only nose2 is in there... I don't see any nose2 flaw here.
It just looks like you have to upgrade indexes.py to use nose2 instead of nose1 skip plugin.

[starsirius]

@thedrow, @aloysbaillet Thanks for the response. Is it possible to skip the tests in mongoengine? I have specified start-dir in the config, and nose2 seems to still discover it. Since it's just a package I installed for my project, (even though it's installed as editable), I think it's reasonable to assume that it's tested already (it should be mongoengine's responsibility to test it). Any ideas?

Thanks a lot. Really appreciate it!

[starsirius]

Hey @thedrow, my directory structure looks like this:

- project/
    - venv/ # My virtualenv
        - src/ # I installed mongoengine and nose2 as editable, so they end up in the src/ folder
            - mongoengine/
                - tests/ # Mongoengine's own tests
                - ...
            - nose2/
        - ...
    - tests/ # My project's tests
        - models/
        - libs/
        - ...
    - project/ # Actual project folder
        - models/
        - libs/
        - ...
    - setup.cfg # Nose2 config here

I will try to move around my own tests folder and see if there is any luck, and please let me know if you have ideas. Thanks a lot for your help! 👍

[aloysbaillet]

With the directory structure it definitely looks like there's a bug in nose2: it should have only been looking in the top level "tests" folder (the start-dir), not in the venv folder...

[starsirius]

@aloysbaillet, thanks for the look up. I've tried to trace the errors, but it's kind of hard for me to spot it. I am sharing what I got so far and maybe you will have a better idea:

• It's not failing for a specific module, but failing only for the first one. The rest of the test modules passed as expected.
• The __import__() function in util.py was actually loading my test module (i.e. "tests.api.client_apps_tests") but for some reason it raises the No module named nose.plugins.skip exception which is actually used in Mongoengine's tests, not my test modules.

This is where I am now, and I have no clue why it tried to load the tests in Mongoengine's own tests in the __import__. 😢

[starsirius]

@thedrow, no problem, I can try that. Thanks for the advise, and I am looking forward to the updates! Thanks again!