Skip to content

Response to record may not have an _original_response attribute #956

@LucHermitte

Description

@LucHermitte

Issue observed

I've run in a scenario where the real response has no _original_response attribute.

I end up observing this kind of of error:

s1tiling/libs/orbit/_providers.py:271: in search
    eofs = self.__dag.search_all(
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/api/core.py:1518: in search_all
    for page_results in self.search_iter_page_plugin(
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/api/core.py:1347: in search_iter_page_plugin
    search_result = self._do_search(
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/api/core.py:1879: in _do_search
    res, nb_res = search_plugin.query(prep, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/plugins/search/qssearch.py:822: in query
    provider_results = self.do_search(prep, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/plugins/search/qssearch.py:1416: in do_search
    return super(ODataV4Search, self).do_search(prep, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/plugins/search/qssearch.py:963: in do_search
    response = self._request(single_search_prep)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/eodag/plugins/search/qssearch.py:1280: in _request
    Response, adapter.build_response(req_prep, urllib_response)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/requests/adapters.py:366: in build_response
    extract_cookies_to_jar(response.cookies, req, resp)
ENVS/s1tiling-otb911-py313/lib/python3.13/site-packages/requests/cookies.py:142: in extract_cookies_to_jar
    jar.extract_cookies(res, req)
ENVS/s1tiling-otb911-py313/lib/python3.13/http/cookiejar.py:1688: in extract_cookies
    for cookie in self.make_cookies(response, request):
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <RequestsCookieJar[]>, response = MockResponse('OK')
request = <requests.cookies.MockRequest object at 0x72e338f8d090>

    def make_cookies(self, response, request):
        """Return sequence of Cookie objects extracted from response object."""
        # get cookie-attributes for RFC 2965 and Netscape protocols
        headers = response.info()
>       rfc2965_hdrs = headers.get_all("Set-Cookie2", [])
                       ^^^^^^^^^^^^^^^
E       AttributeError: 'str' object has no attribute 'get_all'

ENVS/s1tiling-otb911-py313/lib/python3.13/http/cookiejar.py:1604: AttributeError  # the line number may not match b/c I've instrumented the code 

Analysis

We can see that extract_cookies_to_jar does a check,

def extract_cookies_to_jar(jar, request, response):
    """Extract the cookies from the response into a CookieJar.

    :param jar: http.cookiejar.CookieJar (not necessarily a RequestsCookieJar)
    :param request: our own requests.Request object
    :param response: urllib3.HTTPResponse object
    """
    if not (hasattr(response, "_original_response") and response._original_response):  # <<--- HERE
        return
    # the _original_response field is the wrapped httplib.HTTPResponse object,
    req = MockRequest(request)
    # pull out the HTTPMessage with the headers and put it in the mock:
    res = MockResponse(response._original_response.msg)
    jar.extract_cookies(res, req)

However vcrpy always sets self._original_response = self in VCRHTTPResponse.__init__. As a consequence, requests.cookies.extract_cookies_to_jar() will always try to decode cookies, even when there are none to decode.

Possible Fix

The following fix in vcr/stubs/__init__py seems to work:

# in VCRConnection.getreponse()
            # put the response into the cassette
            response = {
                "status": {"code": response.status, "message": response.reason},
                "headers": serialize_headers(response),
                "body": {"string": response_data},
+                "has_original_response": hasattr(response, '_original_response'),
            }
            self.cassette.append(self._vcr_request, response)


# in VCRHTTPResponse.__init__:
-        self._original_response = self  # for requests.session.Session cookie extraction
+        if recorded_response.get("has_original_response", False):
+            self._original_response = self  # for requests.session.Session cookie extraction

~bug

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions