|
Hi all, me again ^^ TL;DR: Is there an easy way to restrict a python unit test module's tests to just a few using their names (e.g. default tags)? I also left a few remarks below about my findings before posting here. We wanted to integrate our Avocado plugin's unit tests with Avocado itself, similarly to the way it is already done in Avocado's own unit tests. So I was looking through Avocado Test Writer’s Guide for examples of this like https://avocado-framework.readthedocs.io/en/91.0/guides/writer/chapters/writing.html#basic-example From what I gathered, I only have to subclass from but I am not sure if this is an error yet or I missed something important in the documentation. AFAIU the stdout/stderr/output checking functionality should only be enabled if one produces files at the level of the test (which I never did). I also read through the caveats section in and while the process isolation does make Finally, the tag restriction seems to be the only restriction possible and perhaps there are some default tags but I couldn't find the example usage anywhere in the documentation due to this warning there: |
Replies: 3 comments 6 replies
|
Hi @clebergnu, @beraldoleal, @willianrampazzo, @ana, does anyone have an idea about the restriction part? I am also now debugging the strange ERROR cases above, I suspect some of the unit test mocking might be causing it as it only happens for this python module. The offensive lines seem to be the following: exists_patch = mock.patch('avocado_i2n.states.setup.os.path.exists',
self.mock_file_exists)
exists_patch.start()
self.addCleanup(exists_patch.stop)not sure how but patching with mock code like this seems to make Avocado expect a |
|
I did a small trace change from the places the mock diff --git a/selftests/test_state_setup.py b/selftests/test_state_setup.py
index 241fcac..7c14958 100755
--- a/selftests/test_state_setup.py
+++ b/selftests/test_state_setup.py
@@ -102,6 +102,12 @@ class StateSetupTest(unittest.TestCase):
self.mock_vms[vm_name].params = self.run_params.object_params(vm_name)
def _file_exists(self, filepath):
+ print(f"called {filepath} from")
+ from traceback import print_stack
+ print_stack()
+ if filepath.endswith(".expected"):
+ print(f"got {filepath}")
+ return False
if self.exist_lambda:
return self.exist_lambda(filepath)
return self.exist_switchand I see the following so it seems that the line ends up in the patched os.path.exists within the tested subpackage module |
|
Hi @pevogam , Based on the name of this discussion ("How to restrict and run standard python unit tests based on name"), is your original question about how to ask Avocado to run *standard Python Unit tests"? Avocado should be able to run any Python unittest. About the reference used, it should be the path to the test file, optionally followed by class and method name. Internally, the Now the |
|
Hi @clebergnu,
I think so, a lot of time has passed since I posted this and eventually I gave up on the idea of trying to restrict to python unit tests. I have shown a diff above of what I replaced to make standard unit tests we used to run directly compatible with avocado where it picks not just the filename but also the particular unit test cases.
Do you have a suggested example command line considering some arbitrary unit test class with multiple test_* cases inside? I could then compare with the previous outcome later on.
It is yes, I think I have reported some issue about it but it was long time ago so will have to refresh more of this in order to be able to say more right now. Will hopefully revisit this topic in the coming months again. |
Yes, the Avocado source directory is full of those examples, and we use Avocado itself to run them. Example: Running one by name: About restricting tests using tags, one could add a tag such as: $ git diff
diff --git a/selftests/unit/test_test.py b/selftests/unit/test_test.py
index 3bf21748c..27aaaae61 100644
--- a/selftests/unit/test_test.py
+++ b/selftests/unit/test_test.py
@@ -49,6 +49,7 @@ class TestClassTestUnit(unittest.TestCase):
self.tmpdir.cleanup()
def test_ugly_name(self):
+ """:avocado: tags=ugly"""
def run(name, path_name):
""" Initialize test and check the dirs were created """
config = {"run.test_runner": 'runner'}And then:
OK! |
|
Makes, sense thanks! As this is a discussion it doesn't have to be closed or opened and hopefully I can revisit this once again the future. |
|
So I could finally refresh this, see below. |
|
I am back and ready to resolve this (hopefully):
The expect file issue is no longer reproducible, must have been fixed in alter iterations before we could catch it. In any case, nobody minds stuff starting to work for free so I guess we are all rather happy about this outcome.
It could be that I am missing something but my difficulty here is mostly about the documentation. While I could easily find an entire "Filtering tests by tags" section at https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html# I did not find even a single example included there about much more regular restricting to file and unit test name. Something like worked well (give or take the fact that it will result in >1 tests if I also have a test named |
|
Ah, regex worked out of the box so telescoped test names worked just fine, splendid! |
I am back and ready to resolve this (hopefully):
The expect file issue is no longer reproducible, must have been fixed in alter iterations before we could catch it. In any case, nobody minds stuff starting to work for free so I guess we are all rather happy about this outcome.
It could be that I am missing something but my difficulty here is mostly about the documentation. While I could easily find an entire "Filtering tests by tags" section at https://avocado-framework.readthedocs.io/en/late…