feat: added pytest fixture to create and use its own temporary directory structure #363

rohansen856 · 2025-05-20T18:13:44Z

What It Does

Fixes #361

Summary

As discussed in issue #361 I have added two files inside the tests directory which contains pytest fixtures to create and use a tmp_dir for the integration tests to run. This is the first part of incrementally implementing the directory isolation functionality for the integration tests.

@zFernand0 If this fix is alright, For the next steps I would like to Implement monkeypatch to temporarily change the Environment variables inside the tmp_dir as to ascertain that the dev's local environment is not clashing with the tests.

How to Test

run the tests using

 pytest ./tests/integration

Review Checklist
I certify that I have:

tested my changes
added/updated automated tests
updated the changelog
followed the contribution guidelines

Additional Comments

…ory structure Signed-off-by: rohansen856 <[email protected]>

zFernand0

This is a great first step! 🥳

I'd love to see this being leveraged by other tests. In order to make this easier to maintain... we could move the creating of the zowe.config.json file in the tmp_path to the isolated_test_env function. Then we could create a class in conftest.py that other test classes can leverage...
Here is an example...

class TestIsolatedEnv(unittest.TestCase):
    """Base class for isolated test environments."""
    @pytest.fixture(autouse=True)
    def _setup_test_env(self, isolated_test_env):
        self.isolated_test_env = isolated_test_env

And then other tests can easily make use of the isolated_test_env by inheriting from it

class TestConsoleIntegration(TestIsolatedEnv):

Also, for easier import, we may need to move conftest.py inside the integration folder (but that may cause other problems 😋
So we might be forced into creating a zowe.test_utils package for this class 😓

The above is just one way (of many) that we could implement this.

zFernand0 · 2025-05-20T21:07:12Z

tests/conftest.py

+    project_dir = tmp_path / "project"
+    global_dir = tmp_path / "global"
+    project_dir.mkdir()
+    global_dir.mkdir()
+
+    return {
+        "project_dir": project_dir,
+        "global_dir": global_dir
+    }


Thanks for keeping the globa vs. project configs here...
I believe that for the purpose of integration testing we could simplify this to just use the
tmp_path / zowe.config.json

Suggested change

project_dir = tmp_path / "project"

global_dir = tmp_path / "global"

project_dir.mkdir()

global_dir.mkdir()

return {

"project_dir": project_dir,

"global_dir": global_dir

}

config_path = Path(__file__).resolve().parent.parent.parent / "zowe.config.json"

config_file = tmp_path / "zowe.config.json"

config_file.write_text(config_path.read_text())

return tmp_path

@zFernand0 @pem70 I think instead of tmp_path we need to return the config_file and config_path variables from the function. It would look something like this:

from pathlib import Path import pytest @pytest.fixture def isolated_test_env(tmp_path): config_path = Path(__file__).resolve().parent.parent.parent / "zowe.config.json" config_file = tmp_path / "zowe.config.json" config_file.write_text(config_path.read_text()) return { "config_path": config_path, "config_file": config_file }

And for the other file (tests/integration/test_integration_env.py):

def test_isolated_env_does_not_touch_real_home(isolated_test_env): config_path = isolated_test_env["config_path"] config_file = isolated_test_env["config_file"] assert config_path.exists(), f"Missing config at {config_path}" assert config_file.exists() assert "profiles" in config_file.read_text()

And yes this simplifies the overall structure by isolating all the logic and assertions in different functions🫡. Please let me know if I am mistaken anywhere...

If this is alright then we can possibly move towards making the structure more modular by using it like this: class TestIsolatedEnv(unittest.TestCase): as you suggested. A step by step approach would be better so I would like to first make a commit for modifications as mentioned above then move towards the modular integration...

@zFernand0 is it ok to require zowe.config.json to be at the root level of the project, or should we allow it to reside at any folder level?

codecov · 2025-05-20T21:10:00Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.05%. Comparing base (d5299bc) to head (fc9df2c).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #363   +/-   ##
=======================================
  Coverage   88.05%   88.05%           
=======================================
  Files          65       65           
  Lines        3340     3340           
=======================================
  Hits         2941     2941           
  Misses        399      399

Flag	Coverage Δ
unittests	`88.05% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

zFernand0 · 2025-05-20T21:10:05Z

Hey @rohansen856,
Please note that all of my comments above are mere suggestions 😋

I'm sure there are better ways to avoid duplicating the creation/copy of the "project" zowe.config.json file into the tmp_path 😋

@t1m0thyj, @traeok, @pem70
I'd love to get your thoughts on this 😋

pem70

LGTM. Same suggestions as @zFernand0 gave.

rohansen856 · 2025-05-22T22:35:55Z

This is a great first step! 🥳

I'd love to see this being leveraged by other tests. In order to make this easier to maintain... we could move the creating of the zowe.config.json file in the tmp_path to the isolated_test_env function. Then we could create a class in conftest.py that other test classes can leverage... Here is an example...
class TestIsolatedEnv(unittest.TestCase):
    """Base class for isolated test environments."""
    @pytest.fixture(autouse=True)
    def _setup_test_env(self, isolated_test_env):
        self.isolated_test_env = isolated_test_env
And then other tests can easily make use of the isolated_test_env by inheriting from it
class TestConsoleIntegration(TestIsolatedEnv):
Also, for easier import, we may need to move conftest.py inside the integration folder (but that may cause other problems 😋 So we might be forced into creating a zowe.test_utils package for this class 😓

The above is just one way (of many) that we could implement this.

Thanks @zFernand0 😁... I tested the functions with the conftest.py file inside /integration folder and it the results were same as before (the fixtures affect them the same way in booth cases), so I think we can definitely move the file as you suggested 🫡.
Got the point of this: class TestConsoleIntegration(TestIsolatedEnv):.
so the conftest.py file would be like this:

class TestIsolatedEnv(unittest.TestCase):
    @pytest.fixture(autouse=True)
    def isolated_test_env(tmp_path):
        config_path = Path(__file__).resolve().parent.parent.parent / "zowe.config.json"

        config_file = tmp_path / "zowe.config.json"
        config_file.write_text(config_path.read_text())

        return {
            "config_path": config_path, 
            "config_file": config_file
        }

and the other file (test_integration_env.py) would be like this:

from .conftest import TestIsolatedEnv


class TestIntegrationEnv(TestIsolatedEnv):
    def test_isolated_env_does_not_touch_real_home(self, isolated_test_env):
        config_path = isolated_test_env["config_path"]
        config_file = isolated_test_env["config_file"]

        self.assertTrue(config_path.exists(), f"Missing config at {config_path}")
        self.assertTrue(config_file.exists())
        self.assertTrue("profiles" in config_file.read_text())

Here, instead of making the test integrations a separate module (which may complicate things unneccesarily), we can use the . notation to import the class (from .conftest import TestIsolatedEnv)🤔...
Thanks for all the suggestions. Lets work on it to find the optimal way🫡.

zFernand0 · 2025-06-02T19:37:36Z

Hey @rohansen856,
Thanks for considering the suggested implementation.
Please let us know when you get a chance to work on those (either here or in a follow-up PR) 😋

rohansen856 · 2025-06-02T20:34:32Z

@zFernand0 I will start working on these from Wednesday at the earliest. I would like to work on this very PR as a draft (a lot more discussions might be ahead and we need to keep track of them in a single thread) and after all the things are finalized I can either request to merge this PR (if the overall structure is still clean at the end) or open up a fresh PR with all the changes in a systematic order...
Thanks!

Signed-off-by: rohansen856 <[email protected]>

…s class Signed-off-by: rohansen856 <[email protected]>

rohansen856 · 2025-06-12T15:57:43Z

@zFernand0 first of all sorry for the late!

I have updated asll the integration test classes with the above discussed method (ex.: TestZosmfIntegration(unittest.TestCase) ->TestZosmfIntegration(TestIsolatedEnv)`) and the tests are seen to be passing!

Copilot

Pull Request Overview

Add pytest-based isolation for integration tests by introducing a temporary zowe.config.json fixture and updating existing tests to use it.

Introduce isolated_test_env fixture and TestIsolatedEnv base class in conftest.py
Update existing integration test classes to inherit from TestIsolatedEnv instead of unittest.TestCase
Add a new test (test_integration_env.py) to verify the isolated environment setup

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/integration/conftest.py	New pytest fixture and base class to set up an isolated test env
tests/integration/test_zosmf.py	Switched from `unittest.TestCase` to `TestIsolatedEnv`
tests/integration/test_zos_tso.py	Switched from `unittest.TestCase` to `TestIsolatedEnv`
tests/integration/test_zos_jobs.py	Switched from `unittest.TestCase` to `TestIsolatedEnv`
tests/integration/test_zos_files.py	Switched from `unittest.TestCase` to `TestIsolatedEnv`
tests/integration/test_zos_console.py	Switched from `unittest.TestCase` to `TestIsolatedEnv`
tests/integration/test_integration_env.py	New test verifying the isolated environment fixture

Comments suppressed due to low confidence (3)

tests/integration/conftest.py:1

The import of unittest isn’t used elsewhere; since pytest fixtures are in use, you can remove this import to simplify dependencies.

import unittest

tests/integration/conftest.py:3

The imported TestCase symbol is not referenced in this file; you can remove it to avoid unused imports.

from unittest import TestCase

tests/integration/test_zosmf.py:3

This import assumes tests/integration is a Python package; ensure there's an __init__.py so integration.conftest can be resolved.

from integration.conftest import TestIsolatedEnv

Copilot · 2025-07-07T15:06:15Z

tests/integration/conftest.py

+        "config_file": config_file
+        }
+
+class TestIsolatedEnv(unittest.TestCase):


Inheriting from unittest.TestCase while using pytest fixtures may lead to unexpected test collection behavior; consider defining TestIsolatedEnv as a plain class or using pytest’s xunit setup hooks instead.

Suggested change

class TestIsolatedEnv(unittest.TestCase):

class TestIsolatedEnv:

I think we should consider this comment as we don't want this to negatively impact test collection behavior. Maybe we can pursue those xunit setup hooks.

Copilot · 2025-07-07T15:06:16Z

tests/integration/conftest.py

+class TestIsolatedEnv(unittest.TestCase):
+    """Base class for isolated test environments."""
+    @pytest.fixture(autouse=True)
+    def _setup_test_env(self, isolated_test_env):
+        self.isolated_test_env = isolated_test_env


Autouse fixtures defined inside a TestCase subclass may not be applied as intended. Move this fixture to the module scope (top level in conftest.py) for pytest to recognize it properly.

Suggested change

class TestIsolatedEnv(unittest.TestCase):

"""Base class for isolated test environments."""

@pytest.fixture(autouse=True)

def _setup_test_env(self, isolated_test_env):

self.isolated_test_env = isolated_test_env

@pytest.fixture(autouse=True)

def _setup_test_env(isolated_test_env):

"""Automatically set up the isolated test environment."""

return isolated_test_env

class TestIsolatedEnv(unittest.TestCase):

"""Base class for isolated test environments."""

def setUp(self):

self.isolated_test_env = _setup_test_env(isolated_test_env)

traeok

Thanks for your work on this @rohansen856. I've left some comments - I believe we should also address the comments posted by Copilot.

traeok · 2025-07-18T17:48:58Z

tests/integration/test_integration_env.py

@@ -0,0 +1,9 @@
+"""Test isolated environment setup for integration tests."""
+
+def test_isolated_env_does_not_touch_real_home(isolated_test_env):


Does this actually verify that the environment does not touch the "real home"?

When you mean "real home," are you referring to the user's home directory? or ZOWE_CLI_HOME?

traeok · 2025-07-18T17:49:55Z

tests/integration/conftest.py

+        "config_file": config_file
+        }
+
+class TestIsolatedEnv(unittest.TestCase):


I think we should consider this comment as we don't want this to negatively impact test collection behavior. Maybe we can pursue those xunit setup hooks.

zFernand0

I really like this PR! 🙏
Just waiting for some of the comments to be address 🙏

feat: added pytest fixture to create and use its own temporary direct…

7a369d5

…ory structure Signed-off-by: rohansen856 <[email protected]>

github-project-automation bot added this to Zowe CLI Squad May 20, 2025

github-project-automation bot moved this to New Issues in Zowe CLI Squad May 20, 2025

zowe-robot moved this from New Issues to In Progress in Zowe CLI Squad May 20, 2025

rohansen856 mentioned this pull request May 20, 2025

[tests]: Make integration tests not dependent on the dev's environment #361

Closed

zFernand0 self-requested a review May 20, 2025 19:13

zFernand0 reviewed May 20, 2025

View reviewed changes

zFernand0 requested review from t1m0thyj and traeok May 20, 2025 21:10

zFernand0 marked this pull request as draft May 20, 2025 21:10

zFernand0 requested a review from pem70 May 22, 2025 13:55

pem70 reviewed May 22, 2025

View reviewed changes

zFernand0 moved this from In Progress to Release Backlog in Zowe CLI Squad Jun 9, 2025

rohansen856 added 4 commits June 11, 2025 01:04

Merge branch 'main' into test-not-dependent-on-env

699a81d

fix: moved conftest file and initialized isolation class

b342ee1

Signed-off-by: rohansen856 <[email protected]>

test: updated isolated env test

7757c2e

Signed-off-by: rohansen856 <[email protected]>

enhancement: created a central class for inheriting unittest Testcase…

ae79b01

…s class Signed-off-by: rohansen856 <[email protected]>

Merge branch 'main' into test-not-dependent-on-env

e2ea9d9

t1m0thyj requested a review from Copilot July 7, 2025 15:05

Copilot AI reviewed Jul 7, 2025

View reviewed changes

traeok reviewed Jul 18, 2025

View reviewed changes

Merge branch 'main' into test-not-dependent-on-env

fc9df2c

zFernand0 self-requested a review August 11, 2025 15:32

zFernand0 reviewed Aug 15, 2025

View reviewed changes

	class TestIsolatedEnv(unittest.TestCase):
	class TestIsolatedEnv:

		@@ -0,0 +1,9 @@
		"""Test isolated environment setup for integration tests."""

		def test_isolated_env_does_not_touch_real_home(isolated_test_env):

feat: added pytest fixture to create and use its own temporary directory structure #363

Are you sure you want to change the base?

feat: added pytest fixture to create and use its own temporary directory structure #363

Uh oh!

Conversation

rohansen856 commented May 20, 2025

Summary

Uh oh!

zFernand0 left a comment

Choose a reason for hiding this comment

Uh oh!

zFernand0 May 20, 2025

Choose a reason for hiding this comment

Uh oh!

rohansen856 May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rohansen856 May 22, 2025

Choose a reason for hiding this comment

Uh oh!

t1m0thyj Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zFernand0 commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pem70 left a comment

Choose a reason for hiding this comment

Uh oh!

rohansen856 commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zFernand0 commented Jun 2, 2025

Uh oh!

rohansen856 commented Jun 2, 2025

Uh oh!

rohansen856 commented Jun 12, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

traeok Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

traeok left a comment

Choose a reason for hiding this comment

Uh oh!

traeok Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

traeok Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

zFernand0 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rohansen856 May 22, 2025 •

edited

Loading

codecov bot commented May 20, 2025 •

edited

Loading

zFernand0 commented May 20, 2025 •

edited

Loading

rohansen856 commented May 22, 2025 •

edited

Loading