Getting a `403 Client Error: Forbidden` error

[EricRibeiro]

Hey 👋

Thank you for your work! I've been trying to use your library to interact with my Smart Lock, but I keep getting the following error:

Traceback (most recent call last):
  File "/Users/Eric/Projects/personal/august/./app.py", line 21, in <module>
    authenticator.send_verification_code()
  File "/opt/homebrew/lib/python3.11/site-packages/yalexs/authenticator.py", line 107, in send_verification_code
    self._api.send_verification_code(
  File "/opt/homebrew/lib/python3.11/site-packages/yalexs/api.py", line 55, in send_verification_code
    return self._dict_to_api(
           ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/yalexs/api.py", line 265, in _dict_to_api
    _raise_response_exceptions(response)
  File "/opt/homebrew/lib/python3.11/site-packages/yalexs/api.py", line 299, in _raise_response_exceptions
    raise err
  File "/opt/homebrew/lib/python3.11/site-packages/yalexs/api.py", line 272, in _raise_response_exceptions
    response.raise_for_status()
  File "/opt/homebrew/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api-production.august.com/validation/email

I have tried using phone instead of email, but it also returns a 403. I wonder if it is related to this or if I'm just doing something wrong. For reference, I am using the code you provided in the README:

from yalexs.api import Api 
from yalexs.authenticator import Authenticator, AuthenticationState
from yalexs.const import Brand

api = Api(timeout=20, brand=Brand.AUGUST)
authenticator = Authenticator(api, "email", "redacted", "redacted")

authentication = authenticator.authenticate()

# State can be either REQUIRES_VALIDATION, BAD_PASSWORD or AUTHENTICATED
# You'll need to call different methods to finish authentication process, see below
state = authentication.state

# If AuthenticationState is BAD_PASSWORD, that means your login_method, username and password do not match

# If AuthenticationState is AUTHENTICATED, that means you're authenticated already. If you specify "access_token_cache_file", the authentication is cached in a file. Everytime you try to authenticate again, it'll read from that file and if you're authenticated already, Authenticator won't call Yale Access again as you have a valid access_token


# If AuthenticationState is REQUIRES_VALIDATION, then you'll need to go through verification process
# send_verification_code() will send a code to either your phone or email depending on login_method
authenticator.send_verification_code()

I see the same Forbidden error if I try authenticating via HomeAssistant:

FWIW, I've bought my first August Smart Lock and created my account this week so my account is fairly new.

[bdraco]

Try tethering to your phone when you do the auth. It may be that the waf doesn't like your ip address and is blocking it

[EricRibeiro]

Unfortunately, it didn't work. I tried it with mine and my partner's phone (she has a different provider), but I'm stuck with the 403.

I have also tried a handful of different servers from the US, Canada, Sweden, Ireland, the UK and a few other countries using NordVPN without success.

[raceconditions]

I've also just set up an account today - I am having exactly the same problem also with August. Tracked down this library, similar to @EricRibeiro, to see if I could play around at a lower level to see what's going on. Same thing for either phone or email when I try to call authenticator.send_verification_code():

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api-production.august.com/validation/phone

[EricRibeiro]

So, I got myself an old Android phone, rooted it and tried following the process described here. Unfortunately, the file was never populated with the key no matter what I tried (using different accounts, triggering auto-lock by moving far enough from the lock, etc.).

After researching, I decided to follow the route described here. I did manage to see the app's requests via mitmproxy, mimic them in Insomnia and get successful responses.

One thing that stood out is the x-august-api-key. Mine is different from what is hard coded in yalexs. However, even if I try replacing the latter with the former, I still get a 403 Forbidden. I suppose this eliminates this header as the cause of the issue. Maybe the problem is in the request body?

I ultimately got the offline key from one particular request captured by mitmproxy, but I still can't get the BLE integration working in HA. It's timing out while trying to connect:

@bdraco, my current problem is now outside the scope of this repo, so feel free to close the issue. I would appreciate any pointers that could help me overcome the error above though. Also, if there is any information I can provide you with to help troubleshoot this 403 error, let me know. I'm happy to help.

[raceconditions]

As a note - I similarly played with this for hours with MITM on the mobile app and also watching the cloud traffic using developer tools in Chrome. In the end, I ended up just hitting the API with basic python and requests. It was Cloudflare blocking the API call with 403. Strangely, for no particular reason, I decided to try "one more time" to enable in HA and it just worked... gave me the prompt in HA to enter the verification code from email.

FWIW - here's what I was getting back from Cloudflare with the 403. I'm also happy to share any additional relevant information, but... was this just transient? Not sure.

<!DOCTYPE html>\n<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->\n<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->\n<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->\n<head>\n<title>Attention Required! | Cloudflare</title>\n<meta charset="UTF-8" />\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />\n<meta http-equiv="X-UA-Compatible" content="IE=Edge" />\n<meta name="robots" content="noindex, nofollow" />\n<meta name="viewport" content="width=device-width,initial-scale=1" />\n<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" />\n<!--[if lt IE 9]><link rel="stylesheet" id=\'cf_styles-ie-css\' href="/cdn-cgi/styles/cf.errors.ie.css" /><![endif]-->\n<style>body{margin:0;padding:0}</style>\n\n\n<!--[if gte IE 10]><!-->\n<script>\n if (!navigator.cookieEnabled) {\n window.addEventListener(\'DOMContentLoaded\', function () {\n var cookieEl = document.getElementById(\'cookie-alert\');\n cookieEl.style.display = \'block\';\n })\n }\n</script>\n<!--<![endif]-->\n\n\n</head>\n<body>\n <div id="cf-wrapper">\n <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" data-translate="enable_cookies">Please enable cookies.</div>\n <div id="cf-error-details" class="cf-error-details-wrapper">\n <div class="cf-wrapper cf-header cf-error-overview">\n <h1 data-translate="block_headline">Sorry, you have been blocked</h1>\n <h2 class="cf-subheadline"><span data-translate="unable_to_access">You are unable to access</span> august.com</h2>\n </div><!-- /.header -->\n\n <div class="cf-section cf-highlight">\n <div class="cf-wrapper">\n <div class="cf-screenshot-container cf-screenshot-full">\n \n <span class="cf-no-screenshot error"></span>\n \n </div>\n </div>\n </div><!-- /.captcha-container -->\n\n <div class="cf-section cf-wrapper">\n <div class="cf-columns two">\n <div class="cf-column">\n <h2 data-translate="blocked_why_headline">Why have I been blocked?</h2>\n\n <p data-translate="blocked_why_detail">This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.</p>\n </div>\n\n <div class="cf-column">\n <h2 data-translate="blocked_resolve_headline">What can I do to resolve this?</h2>\n\n <p data-translate="blocked_resolve_detail">You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.</p>\n </div>\n </div>\n </div><!-- /.section -->\n\n <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300">\n <p class="text-13">\n <span class="cf-footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">####################</strong></span>\n <span class="cf-footer-separator sm:hidden">&bull;</span>\n <span id="cf-footer-item-ip" class="cf-footer-item hidden sm:block sm:mb-1">\n Your IP:\n <button type="button" id="cf-footer-ip-reveal" class="cf-footer-ip-reveal-btn">Click to reveal</button>\n <span class="hidden" id="cf-footer-ip">############</span>\n <span class="cf-footer-separator sm:hidden">&bull;</span>\n </span>\n <span class="cf-footer-item sm:block sm:mb-1"><span>Performance &amp; security by</span> <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing" id="brand_link" target="_blank">Cloudflare</a></span>\n \n </p>\n <script>(function(){function d(){var b=a.getElementById("cf-footer-item-ip"),c=a.getElementById("cf-footer-ip-reveal");b&&"classList"in b&&(b.classList.remove("hidden"),c.addEventListener("click",function(){c.classList.add("hidden");a.getElementById("cf-footer-ip").classList.remove("hidden")}))}var a=document;document.addEventListener&&a.addEventListener("DOMContentLoaded",d)})();</script>\n</div><!-- /.error-footer -->\n\n\n </div><!-- /#cf-error-details -->\n </div><!-- /#cf-wrapper -->\n\n <script>\n window._cf_translation = {};\n \n \n</script>\n\n</body>\n</html>\n

[EricRibeiro]

@raceconditions, thanks for the heads up! I can login now too.

Even with the cloud integration automatically picking up the key, the BLE integration doesn’t work for me. I think the problem is my Raspberry’s Bluetooth. According to this, it doesn’t perform well.

I’ll get myself one of the high performance adaptors in the list and see what happens.

[raceconditions]

@raceconditions, thanks for the heads up! I can login now too.

Even with the cloud integration automatically picking up the key, the BLE integration doesn’t work for me. I think the problem is my Raspberry’s Bluetooth. According to this, it doesn’t perform well.

I’ll get myself one of the high performance adaptors in the list and see what happens.

I'm using a WiFi connect module and all 3 of my locks are working perfectly now. I had considered an ESP32 (I have EverythingOne) BluetoothProxy setup but the wifi connect module seemed a bit more seamless... and it was plug & play with HA once I got through this Cloudflare 403 issue.

[kadlecp]

I'm getting CloudFlare 403 response as well when trying Homebridge Yale Home integration (it's using different library in the background).
It's very strange because /session request works fine but /validation/email request returns 403.
I tried to replicate the requests in Postman and they both work just fine. I was trying to compare the headers but could not find any differences.
Any idea what is triggering CloudFlare 403?

[kadlecp]

The problem is a protection on CF side which is obviously configured by August/Yale.
One problem was a User-Agent, so after changing that I could do some requests, but not all of them.

Then after lots of research I figured that it has something to do with TLS fingerprint used by CF.
Some more details can be found here.
Based on this article I could adapt underlying NodeJS library and it finally started to work.

[bdraco]

That unfortunately didn't solve the issue

[bdraco]

might be fixable with https://github.com/yifeikong/curl_cffi if we know the iphone user agent

[bdraco]

Could do that we likely need to do it just for the login flows and only after they release 0.6.0, so this is still on ice until then