Skip to content

Replace pproxy with in-process mitmproxy in proxy tests - #3483

Open
Fiona Murugi (fMurugi) wants to merge 9 commits into
Qiskit:mainfrom
fMurugi:mitmproxy-test-fixture
Open

Fiona Murugi (fMurugi) wants to merge 9 commits into
Qiskit:mainfrom
fMurugi:mitmproxy-test-fixture

Conversation

@fMurugi

@fMurugi Fiona Murugi (fMurugi) commented Oct 14, 2026 •

Copy link
Copy Markdown

Summary

Replace the pproxy test dependency with mitmproxy, run in-process as a
library rather than as a subprocess, in the proxy integration tests.

Details and comments

pproxy is unmaintained and incompatible with Python 3.14. The proxy
integration tests (test/integration/test_proxies.py) previously shelled
out to the pproxy CLI as a subprocess, blocked direct socket connections,
and verified proxy usage by grep-matching a line out of its stdout log.

This PR swaps pproxy for mitmproxy, embedded directly via its
DumpMaster API on a background thread (not a subprocess). Proxied
requests are now observed directly through a mitmproxy addon callback
instead of parsing log text, which is both more direct and removes the
dependency on an external binary being present on PATH.

Verified by:

  • Running all three proxy integration tests against real IBM Cloud
    credentials ,all passing.
  • Separately confirming the same tests pass under Python 3.14.3 in an
    isolated environment, since Python 3.14 compatibility was the original
    motivation for dropping pproxy.

Fixes #3003

AI/LLM disclosure

  • I didn't use LLM tooling, or only used it privately.
  • I used the following tool to help write this PR description: claude-sonnet-5
  • I used the following tool to generate or modify code: claude-sonnet-5

pproxy is unmaintained and incompatible with Python 3.14. Switch the
proxy integration tests to mitmproxy, run in-process via DumpMaster
on a background thread rather than as a subprocess, and observe
proxied requests directly through an addon callback instead of
parsing CLI log output.
@CLAassistant

CLAassistant commented Oct 14, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - some deviation before reviewing the implementation related to dependencies, as an inline comment.

Comment thread pyproject.toml Outdated

test = [
"ddt>=1.7.2",
"mitmproxy==11.0.2",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the CI run, there might be an extra hurdle.

mitmproxy==11.0.2 depends on a specific version of typing_extensions that conflict with other requirements of other dependencies (mypy, pydantic). It seems to surface in Python 3.10, which we are planning to deprecate (although not in the immediate future).

Can you do a round of investigation on the topic, and specifically, whether mitmproxy relaxes the typing_extensions requirement in some other versions? Ideally we want a solution that works in all versions of Python, and that can co-exist with existing qiskit-ibm-runtime dependencies.

Separately, I'm also curious about your choice of this specific 11.0.2 version? Is it due to requiring a feature introduced or specific to this release?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi I am investing this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, can you comment on your findings?

@fMurugi Fiona Murugi (fMurugi) Oct 19, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello Diego M. Rodríguez (@diego-plan9) after a little investigation i found this out the CI runs the test suite against Python 3.10–3.13 , and there's no single mitmproxy version that supports that whole range — mitmproxy==10.2.4 doesn't support Python 3.11+, and mitmproxy==11.0.2 doesn't support Python 3.10. The environment marker split (python_version < '3.11' / >= '3.11') tells pip which version to install per interpreter, so both ends of the matrix get a working version.

This split can be removed once the project's minimum supported Python version is raised to 3.12+ (mitmproxy has already dropped 3.10/3.11 support in its newer releases), at which point a single modern mitmproxy pin will cover the full supported range again.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch to mitmproxy for proxy tests

3 participants