Describe the bug
Chaining .values_list() or .values() after .select_for_update() silently loses the FOR UPDATE clause: the generated SQL contains no locking clause and no error or warning is raised, so code that relies on the row locks (e.g. a periodic job selecting ids to update, guarded against a concurrent writer) is racy while looking correct.
The cause is visible in tortoise/queryset.py: QuerySet.values_list() and QuerySet.values() build ValuesListQuery / ValuesQuery by passing an explicit list of attributes, and the _select_for_update* family is not among them (the constructors do not even accept it), so the flag set by select_for_update() is dropped instead of being propagated the way _clone() propagates it. Present on 1.1.7 and on current develop.
To Reproduce
import asyncio
from tortoise import Tortoise, fields
from tortoise.models import Model
class Item(Model):
id = fields.IntField(primary_key=True)
name = fields.CharField(max_length=50)
async def main() -> None:
await Tortoise.init(
db_url="postgres://postgres:postgres@localhost:5432/postgres",
modules={"models": ["__main__"]},
)
await Tortoise.generate_schemas()
base = Item.filter(name="x").select_for_update()
print("plain queryset: ", base.sql())
print("values_list(): ", base.values_list("id", flat=True).sql())
print("values(): ", base.values("id").sql())
await Tortoise.close_connections()
asyncio.run(main())
Output (tortoise-orm 1.1.7, PostgreSQL 18, asyncpg):
plain queryset: SELECT "name","id" FROM "item" WHERE "name"=$1 FOR UPDATE
values_list(): SELECT "id" "0" FROM "item" WHERE "name"=$1
values(): SELECT "id" "id" FROM "item" WHERE "name"=$1
Expected behavior
Either of:
values_list() / values() keep the FOR UPDATE clause (SELECT "id" "0" FROM "item" WHERE "name"=$1 FOR UPDATE), matching how limit, offset, distinct, order_by and force_index survive the conversion; or
- the combination raises, like
.values_list() after .only() already does (ValueError(".values_list() cannot be used with .only()")), so the lock can never be lost silently.
Silently dropping a locking clause seems like the worst of the options - we found this only because a pytest-under-PostgreSQL race test failed: a cron doing Model.filter(...).select_for_update().values_list("id", flat=True) inside a transaction did not block a concurrent writer and clobbered its committed update. Workaround we settled on: .select_for_update().only("id") (a plain queryset keeps the clause).
Additional context
Describe the bug
Chaining
.values_list()or.values()after.select_for_update()silently loses theFOR UPDATEclause: the generated SQL contains no locking clause and no error or warning is raised, so code that relies on the row locks (e.g. a periodic job selecting ids to update, guarded against a concurrent writer) is racy while looking correct.The cause is visible in
tortoise/queryset.py:QuerySet.values_list()andQuerySet.values()buildValuesListQuery/ValuesQueryby passing an explicit list of attributes, and the_select_for_update*family is not among them (the constructors do not even accept it), so the flag set byselect_for_update()is dropped instead of being propagated the way_clone()propagates it. Present on1.1.7and on currentdevelop.To Reproduce
Output (tortoise-orm 1.1.7, PostgreSQL 18, asyncpg):
Expected behavior
Either of:
values_list()/values()keep theFOR UPDATEclause (SELECT "id" "0" FROM "item" WHERE "name"=$1 FOR UPDATE), matching howlimit,offset,distinct,order_byandforce_indexsurvive the conversion; or.values_list()after.only()already does (ValueError(".values_list() cannot be used with .only()")), so the lock can never be lost silently.Silently dropping a locking clause seems like the worst of the options - we found this only because a
pytest-under-PostgreSQL race test failed: a cron doingModel.filter(...).select_for_update().values_list("id", flat=True)inside a transaction did not block a concurrent writer and clobbered its committed update. Workaround we settled on:.select_for_update().only("id")(a plain queryset keeps the clause).Additional context
developsources)FOR UPDATE), update_or_create not lock selected row #702 (update_or_createdoes not lock) - this one is specifically about thevalues_list()/values()conversion discarding the flag.