e2e tests are run regardless of --e2e flag on pytest

[NeilHanlon]

While updating to the latest git snapshot for my fedora package review, I found the following occurs when I run pytest, even though from what I can tell, the e2e tests should be skipped if the --e2e flag is not included, but this doesn't appear to be the case in actuality.

Hopefully there's something I'm missing that I'll find after a few hours sleep!

Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.MQqjSu
+ umask 022
+ cd /builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build
+ CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer '
+ export CFLAGS
+ CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer '
+ export CXXFLAGS
+ FFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules '
+ export FFLAGS
+ FCFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules '
+ export FCFLAGS
+ VALAFLAGS=-g
+ export VALAFLAGS
+ RUSTFLAGS='-Copt-level=3 -Cdebuginfo=2 -Ccodegen-units=1 -Cstrip=none -Cforce-frame-pointers=yes -Clink-arg=-specs=/usr/lib/rpm/redhat/redhat-package-notes --cap-lints=warn'
+ export RUSTFLAGS
+ LDFLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes '
+ export LDFLAGS
+ LT_SYS_LIBRARY_PATH=/usr/lib64:
+ export LT_SYS_LIBRARY_PATH
+ CC=gcc
+ export CC
+ CXX=g++
+ export CXX
+ cd pinnwand-b57b6f35e2e4ecd61ad37b4ba18baf106493826e
+ CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer '
+ LDFLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes '
+ PATH=/builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build/BUILDROOT/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
+ PYTHONPATH=/builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build/BUILDROOT/usr/lib64/python3.13/site-packages:/builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build/BUILDROOT/usr/lib/python3.13/site-packages
+ PYTHONDONTWRITEBYTECODE=1
+ PYTEST_ADDOPTS=' --ignore=/builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build/pinnwand-b57b6f35e2e4ecd61ad37b4ba18baf106493826e/.pyproject-builddir'
+ PYTEST_XDIST_AUTO_NUM_WORKERS=12
+ /usr/bin/pytest
============================= test session starts ==============================
platform linux -- Python 3.13.0rc2, pytest-8.3.1, pluggy-1.5.0
rootdir: /builddir/build/BUILD/pinnwand-1.5.0_20240829b57b6f3-build/pinnwand-b57b6f35e2e4ecd61ad37b4ba18baf106493826e
configfile: pyproject.toml
collected 109 items / 1 error
==================================== ERRORS ====================================
__________________________ ERROR collecting test/e2e ___________________________
/usr/lib64/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1387: in _gcd_import
    ???
<frozen importlib._bootstrap>:1360: in _find_and_load
    ???
<frozen importlib._bootstrap>:1331: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:935: in _load_unlocked
    ???
/usr/lib/python3.13/site-packages/_pytest/assertion/rewrite.py:174: in exec_module
    exec(co, module.__dict__)
test/e2e/conftest.py:13: in <module>
    from test.e2e.pageobjects.create_paste_page import CreatePastePage
test/e2e/pageobjects/create_paste_page.py:2: in <module>
    from playwright.sync_api import Page, expect
E   ModuleNotFoundError: No module named 'playwright'

[supakeen]

"Doesn't happen for me". Does this only happen in a RPM build or also outside?

[supakeen]

Oh I might have a hunch, let's see.

[supakeen]

Ok yes, I reproduced it. The tests are not expecting pytest to be ran without installing the dev dependency group hence it is failing during test discovery. I'll figure out a way to disable tests at if the import isn't available during discovery as well.

[NeilHanlon]

I guess in the spec, I can also just drop playwright using sed.. as you said, this may not be a 'supported' scenario.

[supakeen]

Either that or explicitly BuildRequire the dev dependencies until I figure it out, which won't help right now, the e2e tests are currently failing on something that I'm also trying to figure out :)

[supakeen]

@NeilHanlon Can you test on latest master?