DRAFT - Migrate from psycopg2 to psycopg3

[austinweisgrau]

psycopg3 is mostly built as a drop-in upgrade from psycopg2.
Differences are detailed here: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html

A few highlights:

• Cancelling running queries

One major DX improvement is that Ctrl-C can now cancel a running
query. With psycopg2, the python process can be killed but a running
query usually could not be killed (without knowing the PID and running
pg_cancel_backend(pid) from a separate connection), to the immediate
regret of all we who sometimes accidentally execute an unfortunately
large query.

See https://www.psycopg.org/psycopg3/docs/advanced/async.html#interrupting-async-operations

• Breaking changes to parameterization

A few parameterization patterns that worked with psycopg2 no longer
work with psycopg3. They are detailed on the migration guide linked
above, but these in particular should be highlighted:

** You can no longer use IN %s with a tuple

This would work with psycopg2 but will not work with psycopg3:
conn.execute("SELECT * FROM foo WHERE id IN %s", parameters=[(10,20,30)])

Instead, this format is advised:
conn.execute("SELECT * FROM foo WHERE id = ANY(%s)", parameters=[[10,20,30]])

** You can no longer use IS %s
This would work with psycopg2 but will not work with psycopg3:
conn.execute("SELECT * FROM foo WHERE field IS %s", [None])

Instead, IS NOT DISTINCT FROM %s should be used
conn.execute("SELECT * FROM foo WHERE field IS NOT DISTINCT FROM %s", [None])

• Breaking changes to psycopg2 copy methods

copy methods are refactored and consolidated in psycopg3. See the
migration guide for details.

[shaunagm]

Looks like there are some file conflicts? Also, I thought we'd fixed it so the tests would get run against PRs to major-release?

If we can get this to run the tests, and the tests pass, we can merge. Though I will do so warily regardless since psycopg has historically caused a huge amount of installation errors. Actually it'd be great if we could test this against an M1 mac, @austinweisgrau do you happen to have one you could test manually?

[shaunagm]

I'm responding with 'request changes' because we need to figure out how to get the tests run on this (see more detail in other comment)

[austinweisgrau]

I believe the major release branch needs to be brought up to date and that should simplify the diffs considerably

[austinweisgrau]

Ahh it needs more configuration anyways to work propertly, making this a DRAFT for now