[Potential BUG] Unexpected error in test suite: `test_type_files_included_by_default`

[abravalheri]

click to see failures

___________________________ TestTypeInfoFiles.test_type_files_included_by_default[namespace_nested_inside_regular-dont_include_package_data] ___________________________
[gw1] linux -- Python 3.9.14 /home/jgerity/repos/setuptools.git/.direnv/python-3.9.14/bin/python3

self = <setuptools.tests.test_build_py.TestTypeInfoFiles object at 0x72a9790e64c0>, tmpdir_cwd = local('/home/jgerity/repos/setuptools.git')
pyproject = 'dont_include_package_data', example = 'namespace_nested_inside_regular' 

    @pytest.mark.parametrize(
        "pyproject", ["default_pyproject", "dont_include_package_data"]
    )
    @pytest.mark.parametrize("example", EXAMPLES.keys())
    def test_type_files_included_by_default(self, tmpdir_cwd, pyproject, example):
        structure = {
            **self.EXAMPLES[example]["directory_structure"],
            "pyproject.toml": self.PYPROJECTS[pyproject],
        }
        expected_type_files = self.EXAMPLES[example]["expected_type_files"]
        jaraco.path.build(structure)
     
        build_py = get_finalized_build_py()
        outputs = get_outputs(build_py)
>       assert expected_type_files <= outputs
E       AssertionError: assert {'foo/__init_...foo/py.typed'} <= set()
E         
E         Extra items in the left set:
E         'foo/__init__.pyi'
E         'foo/py.typed'
E         'foo/namespace/foo.pyi'

/home/jgerity/repos/setuptools.git/setuptools/tests/test_build_py.py:412: AssertionError

test_type_files_included_by_default previously passed when parametrized by dont_include_package_data, because the associated metadata had the tools.setuptools typo and include-package-data was not being disabled as a result. Fixing the metadata causes the test to fail with this parameter, presumably because the typing files are considered package data.

...

Interestingly, I am having difficulty reproducing the problem I thought was here in order to file a separate issue. In other words, in an attempted reproduction, if I set include-package-data = false, I am still getting the typing files in my resulting wheel.

...

I am at this point confident that the problem is exclusive to the test suite. It is kinda tricky to figure out exactly how the tests should be tweaked to properly test this feature, but at least this isn't a new bug in setuptools

I believe the crux of the problem is that the build_py used in the test has an unpopulated packages attribute. If I manually step into the failing test and call build_py._get_pkg_data_files(appropriate_package_name) I can see the typing files being collected.

But I'm having difficulty understanding the difference in what the tests do and what a pip install does, where the two differ, and how to resolve that so that the tests are closer to the actual functionality.

...

Originally posted by @SnoopJ in #4151 (comment)

---

I have also found that the feature seems to work in practice, but doesn't seem to work in the tests. The tests pass if I change

        build_py = get_finalized_build_py()
        outputs = get_outputs(build_py)

to

        build_py = get_finalized_build_py()
        build_py.run_command("build")
        outputs = get_outputs(build_py)

I don't really understand what all the different possible arguments to run_command do, so I am not very sure why this works, but it does seem to work.

Originally posted by @Danie-1 in #4151 (comment)