"Upgrade Required" (response code 426) when using urequests for an API (MongoDB Data API), likely due to HTTP 1.1+ being required

[sgbaird]

For the last year or so, I've been using Pico W's to upload data directly to a MongoDB Atlas database using the Atlas Data API; however I recently started getting "426: Upgrade Required" as the response (even with a completely fresh MongoDB Atlas M0 cluster). I verified that I can use the Data API through Python:

import requests
import json
url = "https://us-east-2.aws.data.mongodb-api.com/app/data-<APP_ID>/endpoint/data/v1/action/insertOne"

payload = json.dumps({
    "collection": "write-to-me",
    "database": "test-db",
    "dataSource": "Cluster0",
    "projection": {
        "_id": 1
    }
})
headers = {
  'Content-Type': 'application/json',
  'Access-Control-Request-Headers': '*',
  'api-key': '<API_KEY>,
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

However, I get the "Upgrade Required" response mentioned above when running MicroPython on the Pico W. I've tried v1.2.1, v1.22.2, and "latest nightly preview build" 1.23.*. The two stable versions led to the same error, while urequests on the nightly preview seems to be broken altogether.

The Atlas Data API specs require HTTP 1.1+, and it seems that urequests on the Pico W may only support HTTP 1.0 (see also https://forums.raspberrypi.com/viewtopic.php?t=339080). This seems likely based on the issue I'm experiencing. This is urgent for me, since I have tutorials, videos, and class assignments that depend on this approach. I understand that the change happened on MongoDB's side, but I'd really like to find a way to get the Pico W communicating with the MongoDB Atlas Data API again.

I have some credentials I've made public for the course (only used for public demo purposes). The following script should be fully self-contained and runnable.

# based on https://medium.com/@johnlpage/introduction-to-microcontrollers-and-the-pi-pico-w-f7a2d9ad1394
from netman import connectWiFi
import urequests

from my_secrets import SSID, PASSWORD, COURSE_ID

connectWiFi(SSID, PASSWORD, country="US")

DATA_API_KEY = "UT4cdinBetBaNqCBc5hISkaArhllv5dWfzXgbYsLYzpv79nqNhVwVsudQU5ZUmBE"  # Public API key for demo purposes only
CLUSTER_NAME = "test-cluster"
DATABASE_NAME = "test-db"
COLLECTION_NAME = "write-to-me"

ENDPOINT_BASE_URL = (
    "https://us-east-2.aws.data.mongodb-api.com/app/data-ibmqs/endpoint/data/v1"
)

endpoint_url = f"{ENDPOINT_BASE_URL}/action/insertOne"

headers = {"api-key": DATA_API_KEY}
document = {"course_id": COURSE_ID}

payload = {
    "dataSource": CLUSTER_NAME,
    "database": DATABASE_NAME,
    "collection": COLLECTION_NAME,
    "document": document,
}

print(f"sending document to {CLUSTER_NAME}:{DATABASE_NAME}:{COLLECTION_NAME}")

num_retries = 3
for _ in range(num_retries):
    response = urequests.post(endpoint_url, headers=headers, json=payload)
    txt = str(response.text)
    status_code = response.status_code

    print(f"Response: ({status_code}), msg = {txt}")

    response.close()

    if status_code == 201:
        print("Added Successfully")
        break

    print("Retrying...")

The response used to be:

MAC address: ***
connected
ip = ***
sending document to test-cluster:test-db:write-to-me
Response: (201), msg = {"insertedId":"6594bfbfb3c925d2fdfbb7e8"}
Added Successfully

Now, the response is:

MAC address: ***
connected
ip = ***
sending document to test-cluster:test-db:write-to-me
Response: (426), msg = Upgrade Required
Retrying...
Response: (426), msg = Upgrade Required
Retrying...
Response: (426), msg = Upgrade Required
Retrying...

[Josverl]

See issue micropython/micropython-lib#844

Until there is a permanent fix you could copy/patch your own requests11.py module so see if that is good enough for your scenario

[sgbaird]

The workaround of a patched version (somewhat poor naming on my part as urequests_2.py), but the MongoDB Atlas Data API is now deprecated 🙃. So, two birds with one stone by removing my dependence on the Data API 😅, though I'm sure there are others who require the update.