I doubt this is a bug, probably an implementation gap.
But it was very counterintuitive to me.
Using request(..., impersonate='chrome') or some other native target string eventually winds up calling lib.curl_easy_impersonate(), which controls headers itself.
Using request(..., impersonate=some_fingerprint) where some_fingerprint is a Fingerprint() object sets all the curl options right in the python code based on the attributes of the Fingerprint object itself.
However, the native Fingerprint() objects that come from the open source code don't contain a single header. So, the following code block will include real-looking headers with the request:
print(curl_cffi.requests.get('http://httpbin.org/headers', impersonate='chrome').text)
but the following will not:
fp = curl_cffi.FingerprintManager().get_fingerprint('chrome146')
print(curl_cffi.requests.get('http://httpbin.org/headers', impersonate=fp).text)
For our purposes, a large, up-to-date set of realistic fingerprints is beneficial, so I subscribed our company to the impersonate.pro paid plan to get all the header data. There are now more fingerprints that do have populated headers attributes.
Interestingly, the original fingerprints without headers are still included. So, doing something like choosing a random fingerprint via
fp_manager = curl_cffi.FingerprintManager()
fp_names = [fp_['name'] for fp_ in fp_manager.list_fingerprints()]
fp_name = random.choice(fp_names)
fp = curl_cffi.get_fingerprint(fp_name)
print(curl_cffi.requests.get('http://httpbin.org/headers', impersonate=fp).text)
will occasionally make requests that have no default headers set, not because the default_headers arg is set incorrectly, but because the default fingerprints are still there and have no headers.
Obviously, I can filter out those choices, but in this case it seems that the library is working against me.
Perhaps this limitation is well-documented somewhere and I simply missed it, otherwise, please accept this feedback.
Using curl_cffi 0.16.0 on python 3.12
I doubt this is a bug, probably an implementation gap.
But it was very counterintuitive to me.
Using request(..., impersonate='chrome') or some other native target string eventually winds up calling lib.curl_easy_impersonate(), which controls headers itself.
Using request(..., impersonate=some_fingerprint) where some_fingerprint is a Fingerprint() object sets all the curl options right in the python code based on the attributes of the Fingerprint object itself.
However, the native Fingerprint() objects that come from the open source code don't contain a single header. So, the following code block will include real-looking headers with the request:
but the following will not:
For our purposes, a large, up-to-date set of realistic fingerprints is beneficial, so I subscribed our company to the impersonate.pro paid plan to get all the header data. There are now more fingerprints that do have populated headers attributes.
Interestingly, the original fingerprints without headers are still included. So, doing something like choosing a random fingerprint via
will occasionally make requests that have no default headers set, not because the
default_headersarg is set incorrectly, but because the default fingerprints are still there and have no headers.Obviously, I can filter out those choices, but in this case it seems that the library is working against me.
Perhaps this limitation is well-documented somewhere and I simply missed it, otherwise, please accept this feedback.
Using curl_cffi 0.16.0 on python 3.12