Skip to content

.values_list() / .values() silently drop select_for_update() - rows are not locked #2250

Description

@dgavrilov

Describe the bug

Chaining .values_list() or .values() after .select_for_update() silently loses the FOR UPDATE clause: the generated SQL contains no locking clause and no error or warning is raised, so code that relies on the row locks (e.g. a periodic job selecting ids to update, guarded against a concurrent writer) is racy while looking correct.

The cause is visible in tortoise/queryset.py: QuerySet.values_list() and QuerySet.values() build ValuesListQuery / ValuesQuery by passing an explicit list of attributes, and the _select_for_update* family is not among them (the constructors do not even accept it), so the flag set by select_for_update() is dropped instead of being propagated the way _clone() propagates it. Present on 1.1.7 and on current develop.

To Reproduce

import asyncio

from tortoise import Tortoise, fields
from tortoise.models import Model


class Item(Model):
    id = fields.IntField(primary_key=True)
    name = fields.CharField(max_length=50)


async def main() -> None:
    await Tortoise.init(
        db_url="postgres://postgres:postgres@localhost:5432/postgres",
        modules={"models": ["__main__"]},
    )
    await Tortoise.generate_schemas()

    base = Item.filter(name="x").select_for_update()
    print("plain queryset: ", base.sql())
    print("values_list():  ", base.values_list("id", flat=True).sql())
    print("values():       ", base.values("id").sql())

    await Tortoise.close_connections()


asyncio.run(main())

Output (tortoise-orm 1.1.7, PostgreSQL 18, asyncpg):

plain queryset:  SELECT "name","id" FROM "item" WHERE "name"=$1 FOR UPDATE
values_list():   SELECT "id" "0" FROM "item" WHERE "name"=$1
values():        SELECT "id" "id" FROM "item" WHERE "name"=$1

Expected behavior

Either of:

  1. values_list() / values() keep the FOR UPDATE clause (SELECT "id" "0" FROM "item" WHERE "name"=$1 FOR UPDATE), matching how limit, offset, distinct, order_by and force_index survive the conversion; or
  2. the combination raises, like .values_list() after .only() already does (ValueError(".values_list() cannot be used with .only()")), so the lock can never be lost silently.

Silently dropping a locking clause seems like the worst of the options - we found this only because a pytest-under-PostgreSQL race test failed: a cron doing Model.filter(...).select_for_update().values_list("id", flat=True) inside a transaction did not block a concurrent writer and clobbered its committed update. Workaround we settled on: .select_for_update().only("id") (a plain queryset keeps the clause).

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions