Repository navigation
Replace pproxy with in-process mitmproxy in proxy tests - #3483
Fiona Murugi (fMurugi) wants to merge 9 commits into
Conversation
pproxy is unmaintained and incompatible with Python 3.14. Switch the proxy integration tests to mitmproxy, run in-process via DumpMaster on a background thread rather than as a subprocess, and observe proxied requests directly through an addon callback instead of parsing CLI log output.
Diego M. Rodríguez (diego-plan9)
left a comment
There was a problem hiding this comment.
Thanks - some deviation before reviewing the implementation related to dependencies, as an inline comment.
|
|
||
| test = [ | ||
| "ddt>=1.7.2", | ||
| "mitmproxy==11.0.2", |
There was a problem hiding this comment.
Looking at the CI run, there might be an extra hurdle.
mitmproxy==11.0.2 depends on a specific version of typing_extensions that conflict with other requirements of other dependencies (mypy, pydantic). It seems to surface in Python 3.10, which we are planning to deprecate (although not in the immediate future).
Can you do a round of investigation on the topic, and specifically, whether mitmproxy relaxes the typing_extensions requirement in some other versions? Ideally we want a solution that works in all versions of Python, and that can co-exist with existing qiskit-ibm-runtime dependencies.
Separately, I'm also curious about your choice of this specific 11.0.2 version? Is it due to requiring a feature introduced or specific to this release?
There was a problem hiding this comment.
Hi I am investing this.
There was a problem hiding this comment.
Thanks, can you comment on your findings?
There was a problem hiding this comment.
hello Diego M. Rodríguez (@diego-plan9) after a little investigation i found this out the CI runs the test suite against Python 3.10–3.13 , and there's no single mitmproxy version that supports that whole range — mitmproxy==10.2.4 doesn't support Python 3.11+, and mitmproxy==11.0.2 doesn't support Python 3.10. The environment marker split (python_version < '3.11' / >= '3.11') tells pip which version to install per interpreter, so both ends of the matrix get a working version.
This split can be removed once the project's minimum supported Python version is raised to 3.12+ (mitmproxy has already dropped 3.10/3.11 support in its newer releases), at which point a single modern mitmproxy pin will cover the full supported range again.
…roxy-test-fixture
Summary
Replace the
pproxytest dependency withmitmproxy, run in-process as alibrary rather than as a subprocess, in the proxy integration tests.
Details and comments
pproxyis unmaintained and incompatible with Python 3.14. The proxyintegration tests (
test/integration/test_proxies.py) previously shelledout to the
pproxyCLI as a subprocess, blocked direct socket connections,and verified proxy usage by grep-matching a line out of its stdout log.
This PR swaps
pproxyformitmproxy, embedded directly via itsDumpMasterAPI on a background thread (not a subprocess). Proxiedrequests are now observed directly through a mitmproxy addon callback
instead of parsing log text, which is both more direct and removes the
dependency on an external binary being present on
PATH.Verified by:
credentials ,all passing.
isolated environment, since Python 3.14 compatibility was the original
motivation for dropping
pproxy.Fixes #3003
AI/LLM disclosure